User interface for status indicators

ABSTRACT

The present disclosure generally relates to managing audio messages. In some examples, user interfaces for displaying visual indications are described.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationSer. No. 63/041,979, filed on Jun. 21, 2020, titled “USER INTERFACE FORAUDIO MESSAGE,” and U.S. Provisional Patent Application Ser. No.63/023,225, filed on May 11, 2020, titled “USER INTERFACE FOR AUDIOMESSAGE,” the contents of which are hereby incorporated by reference intheir entireties.

FIELD

The present disclosure relates generally to computer user interfaces,and more specifically to techniques for managing audio messages.

BACKGROUND

Users send instant messages to communicate with other users usingalphanumeric text. Such messaging techniques typically require the userto enter text using a keyboard or for a device to transcribe audio intotext to be subsequently sent. Users interact with external devices usingmobile devices. Such interactions often are cumbersome and require theuser to interact with multiple external devices individually.

BRIEF SUMMARY

Current techniques for managing audio messages using electronic devicesare generally cumbersome and inefficient. For example, some existingtechniques use a complex and time-consuming user interface, which mayinclude multiple key presses or keystrokes. Existing techniques requiremore time than necessary, wasting user time and device energy. Thislatter consideration is particularly important in battery-operateddevices.

Accordingly, the present technique provides electronic devices withfaster, more efficient methods and interfaces for managing audiomessages. Such methods and interfaces optionally complement or replaceother methods for managing messages. Such methods and interfaces reducethe cognitive burden on a user and produce a more efficienthuman-machine interface. For battery-operated computing devices, suchmethods and interfaces conserve power and increase the time betweenbattery charges. Such methods and interfaces enable messages to betransmitted that include the tone of the message (e.g., importance,seriousness, sarcastic, sad, cheerful), rather than just the words ofthe message, and enable the recipient to recognize the speaker of themessage by the speaker's own voice. Such methods and interfaces alsoreduce or eliminate the need to provide input using a keyboard or tocorrect a transcription using input.

Current techniques for managing statuses of external devices at a mobiledevice are generally cumbersome and inefficient. For example, someexisting techniques use a complex and time-consuming user interface,which may include multiple key presses or keystrokes. Existingtechniques require more time than necessary, wasting user time anddevice energy. This latter consideration is particularly important inbattery-operated devices.

Accordingly, the present technique provides electronic devices withfaster, more efficient methods and interfaces for managing externaldevices. Such methods and interfaces optionally complement or replaceother methods for managing external devices, such as allowing a user tomanage external devices individual or managing external devices at thesame time via a single input. Such methods and interfaces reduce thecognitive burden on a user and produce a more efficient human-machineinterface. For battery-operated computing devices, such methods andinterfaces conserve power and increase the time between battery charges.

In accordance with some embodiments, a method is described. The methodis performed at a computer system that is in communication with adisplay generation component. The method comprises: displaying, via thedisplay generation component, a first user interface that includes: afirst selectable user interface object, wherein selection of the firstselectable user interface object initiates a process for recording andsending a first audio message to a set of external devices, wherein theset of external devices includes a first set of one or more externaldevices and a second set of one or more external devices; and a firstaccessory control user interface object corresponding to a firstexternal accessory device, wherein selection of the first accessorycontrol user interface object initiates a process to transmit aninstruction to change a state of the first external accessory device;while displaying the first user interface, receiving an input thatcorresponds to a request to display a second user interface; and inresponse to receiving the request to display the second user interface,displaying the second user interface, wherein the second user interfaceincludes: a second selectable user interface object, wherein selectionof the second selectable user interface object initiates a process forrecording and sending a second audio message to the second set of one ormore external devices without sending the second audio message to thefirst set of one or more external devices; and a second accessorycontrol user interface object corresponding to a second externalaccessory device, wherein selection of the second accessory control userinterface object initiates a process to transmit an instruction tochange a state of the second external accessory device.

In accordance with some embodiments, a non-transitory computer-readablestorage medium is described. The non-transitory computer-readablestorage medium stores one or more programs configured to be executed byone or more processors of a computer system in communication with adisplay generation component, the one or more programs includinginstructions for: displaying, via the display generation component, afirst user interface that includes: a first selectable user interfaceobject, wherein selection of the first selectable user interface objectinitiates a process for recording and sending a first audio message to aset of external devices, wherein the set of external devices includes afirst set of one or more external devices and a second set of one ormore external devices; and a first accessory control user interfaceobject corresponding to a first external accessory device, whereinselection of the first accessory control user interface object initiatesa process to transmit an instruction to change a state of the firstexternal accessory device; while displaying the first user interface,receiving an input that corresponds to a request to display a seconduser interface; and in response to receiving the request to display thesecond user interface, displaying the second user interface, wherein thesecond user interface includes: a second selectable user interfaceobject, wherein selection of the second selectable user interface objectinitiates a process for recording and sending a second audio message tothe second set of one or more external devices without sending thesecond audio message to the first set of one or more external devices;and a second accessory control user interface object corresponding to asecond external accessory device, wherein selection of the secondaccessory control user interface object initiates a process to transmitan instruction to change a state of the second external accessorydevice.

In accordance with some embodiments, a transitory computer-readablestorage medium is described. The transitory computer-readable storagemedium stores one or more programs configured to be executed by one ormore processors of a computer system in communication with a displaygeneration component, the one or more programs including instructionsfor: displaying, via the display generation component, a first userinterface that includes: a first selectable user interface object,wherein selection of the first selectable user interface objectinitiates a process for recording and sending a first audio message to aset of external devices, wherein the set of external devices includes afirst set of one or more external devices and a second set of one ormore external devices; and a first accessory control user interfaceobject corresponding to a first external accessory device, whereinselection of the first accessory control user interface object initiatesa process to transmit an instruction to change a state of the firstexternal accessory device; while displaying the first user interface,receiving an input that corresponds to a request to display a seconduser interface; and in response to receiving the request to display thesecond user interface, displaying the second user interface, wherein thesecond user interface includes: a second selectable user interfaceobject, wherein selection of the second selectable user interface objectinitiates a process for recording and sending a second audio message tothe second set of one or more external devices without sending thesecond audio message to the first set of one or more external devices;and a second accessory control user interface object corresponding to asecond external accessory device, wherein selection of the secondaccessory control user interface object initiates a process to transmitan instruction to change a state of the second external accessorydevice.

In accordance with some embodiments, a computer system is described. Thecomputer system comprises: one or more processors, wherein the computersystem is in communication with a display generation component; andmemory storing one or more programs configured to be executed by the oneor more processors, the one or more programs including instructions for:displaying, via the display generation component, a first user interfacethat includes: a first selectable user interface object, whereinselection of the first selectable user interface object initiates aprocess for recording and sending a first audio message to a set ofexternal devices, wherein the set of external devices includes a firstset of one or more external devices and a second set of one or moreexternal devices; and a first accessory control user interface objectcorresponding to a first external accessory device, wherein selection ofthe first accessory control user interface object initiates a process totransmit an instruction to change a state of the first externalaccessory device; while displaying the first user interface, receivingan input that corresponds to a request to display a second userinterface; and in response to receiving the request to display thesecond user interface, displaying the second user interface, wherein thesecond user interface includes: a second selectable user interfaceobject, wherein selection of the second selectable user interface objectinitiates a process for recording and sending a second audio message tothe second set of one or more external devices without sending thesecond audio message to the first set of one or more external devices;and a second accessory control user interface object corresponding to asecond external accessory device, wherein selection of the secondaccessory control user interface object initiates a process to transmitan instruction to change a state of the second external accessorydevice.

In accordance with some embodiments, a computer system is described. Thecomputer system comprising: means for displaying, via a displaygeneration component that is in communication with the computer system,a first user interface that includes: a first selectable user interfaceobject, wherein selection of the first selectable user interface objectinitiates a process for recording and sending a first audio message to aset of external devices, wherein the set of external devices includes afirst set of one or more external devices and a second set of one ormore external devices; and a first accessory control user interfaceobject corresponding to a first external accessory device, whereinselection of the first accessory control user interface object initiatesa process to transmit an instruction to change a state of the firstexternal accessory device; means, while displaying the first userinterface, for receiving an input that corresponds to a request todisplay a second user interface; and means, responsive to receiving therequest to display the second user interface, for displaying the seconduser interface, wherein the second user interface includes: a secondselectable user interface object, wherein selection of the secondselectable user interface object initiates a process for recording andsending a second audio message to the second set of one or more externaldevices without sending the second audio message to the first set of oneor more external devices; and a second accessory control user interfaceobject corresponding to a second external accessory device, whereinselection of the second accessory control user interface objectinitiates a process to transmit an instruction to change a state of thesecond external accessory device.

In accordance with some embodiments, a method is described. The methodis performed at a computer system in communication with a displaygeneration component. The method comprises: receiving first datacorresponding to first media; displaying, via the display generationcomponent, until an end of a first time period, a first notificationwith an indication of the first media; after receiving the first data,receiving second data corresponding to second media, wherein the secondmedia is different from the first media; and in accordance with adetermination that the second media meets a set of one or more criteriaand in accordance with a determination based on (1) a time that thesecond data was received and (2) a time that the first data wasreceived, displaying, via the display generation component, until an endof a second time period, a second notification with an indication of thefirst media, wherein an end of the first time period is different froman end of the second time period.

In accordance with some embodiments, a non-transitory computer-readablestorage medium is described. The non-transitory computer-readablestorage medium stores one or more programs configured to be executed byone or more processors of a computer system in communication with adisplay generation component, the one or more programs includinginstructions for: receiving first data corresponding to first media;displaying, via the display generation component, until an end of afirst time period, a first notification with an indication of the firstmedia; after receiving the first data, receiving second datacorresponding to second media, wherein the second media is differentfrom the first media; and in accordance with a determination that thesecond media meets a set of one or more criteria and in accordance witha determination based on (1) a time that the second data was receivedand (2) a time that the first data was received, displaying, via thedisplay generation component, until an end of a second time period, asecond notification with an indication of the first media, wherein anend of the first time period is different from an end of the second timeperiod.

In accordance with some embodiments, a transitory computer-readablestorage medium is described. The transitory computer-readable storagemedium stores one or more programs configured to be executed by one ormore processors of a computer system in communication with a displaygeneration component, the one or more programs including instructionsfor: receiving first data corresponding to first media; displaying, viathe display generation component, until an end of a first time period, afirst notification with an indication of the first media; afterreceiving the first data, receiving second data corresponding to secondmedia, wherein the second media is different from the first media; andin accordance with a determination that the second media meets a set ofone or more criteria and in accordance with a determination based on (1)a time that the second data was received and (2) a time that the firstdata was received, displaying, via the display generation component,until an end of a second time period, a second notification with anindication of the first media, wherein an end of the first time periodis different from an end of the second time period.

In accordance with some embodiments, a computer system is described. Thecomputer system comprises: one or more processors, wherein the computersystem is in communication with a display generation component; andmemory storing one or more programs configured to be executed by the oneor more processors, the one or more programs including instructions for:receiving first data corresponding to first media; displaying, via thedisplay generation component, until an end of a first time period, afirst notification with an indication of the first media; afterreceiving the first data, receiving second data corresponding to secondmedia, wherein the second media is different from the first media; andin accordance with a determination that the second media meets a set ofone or more criteria and in accordance with a determination based on (1)a time that the second data was received and (2) a time that the firstdata was received, displaying, via the display generation component,until an end of a second time period, a second notification with anindication of the first media, wherein an end of the first time periodis different from an end of the second time period.

In accordance with some embodiments, a computer system is described. Thecomputer system comprises: means for receiving first data correspondingto first media; means for displaying, via a generation component that isin communication with the computer system, until an end of a first timeperiod, a first notification with an indication of the first media;means, after receiving the first data, for receiving second datacorresponding to second media, wherein the second media is differentfrom the first media; and means, in accordance with a determination thatthe second media meets a set of one or more criteria and in accordancewith a determination based on (1) a time that the second data wasreceived and (2) a time that the first data was received, fordisplaying, via the display generation component, until an end of asecond time period, a second notification with an indication of thefirst media, wherein an end of the first time period is different froman end of the second time period.

In accordance with some embodiments, a method is described. The methodis performed at an electronic device having one or more microphones, oneor more speakers, and one or more display devices. The method comprises:receiving voice input via the one or more microphones; and displaying avisual indication of the voice input, wherein: in accordance with adetermination that a portion of the voice input corresponds to voiceinput that is to be transmitted to one or more devices displaying, viathe one or more display devices, the visual indication includesdisplaying the visual indication with a first set of one or more colors;and in accordance with a determination that the voice input does notinclude an instruction to transmit any portion of the voice input toother devices, displaying, via the one or more display devices, thevisual indication includes displaying the visual indication with asecond set of one or more colors that is different from the first set ofone or more colors.

In accordance with some embodiments, a non-transitory computer-readablestorage medium is described. The non-transitory computer-readablestorage medium stores one or more programs configured to be executed byone or more processors of an electronic device having one or moremicrophones, one or more speakers, and one or more display devices, theone or more programs including instructions for: receiving voice inputvia the one or more microphones; and displaying a visual indication ofthe voice input, wherein: in accordance with a determination that aportion of the voice input corresponds to voice input that is to betransmitted to one or more devices displaying, via the one or moredisplay devices, the visual indication includes displaying the visualindication with a first set of one or more colors; and in accordancewith a determination that the voice input does not include aninstruction to transmit any portion of the voice input to other devices,displaying, via the one or more display devices, the visual indicationincludes displaying the visual indication with a second set of one ormore colors that is different from the first set of one or more colors.

In accordance with some embodiments, a transitory computer-readablestorage medium is described. The transitory computer-readable storagemedium stores one or more programs configured to be executed by one ormore processors of an electronic device having one or more microphones,one or more speakers, and one or more display devices, the one or moreprograms including instructions for: receiving voice input via the oneor more microphones; and displaying a visual indication of the voiceinput, wherein: in accordance with a determination that a portion of thevoice input corresponds to voice input that is to be transmitted to oneor more devices displaying, via the one or more display devices, thevisual indication includes displaying the visual indication with a firstset of one or more colors; and in accordance with a determination thatthe voice input does not include an instruction to transmit any portionof the voice input to other devices, displaying, via the one or moredisplay devices, the visual indication includes displaying the visualindication with a second set of one or more colors that is differentfrom the first set of one or more colors.

In accordance with some embodiments, an electronic device is described.The electronic device comprises: one or more microphones; one or morespeakers; one or more display devices; one or more processors; andmemory storing one or more programs configured to be executed by the oneor more processors, the one or more programs including instructions for:receiving voice input via the one or more microphones; and displaying avisual indication of the voice input, wherein: in accordance with adetermination that a portion of the voice input corresponds to voiceinput that is to be transmitted to one or more devices displaying, viathe one or more display devices, the visual indication includesdisplaying the visual indication with a first set of one or more colors;and in accordance with a determination that the voice input does notinclude an instruction to transmit any portion of the voice input toother devices, displaying, via the one or more display devices, thevisual indication includes displaying the visual indication with asecond set of one or more colors that is different from the first set ofone or more colors.

In accordance with some embodiments, an electronic device is described.The electronic device comprises: one or more microphones; one or morespeakers; one or more display devices; means for receiving voice inputvia the one or more microphones; and means displaying a visualindication of the voice input, wherein: in accordance with adetermination that a portion of the voice input corresponds to voiceinput that is to be transmitted to one or more devices displaying, viathe one or more display devices, the visual indication includesdisplaying the visual indication with a first set of one or more colors;and in accordance with a determination that the voice input does notinclude an instruction to transmit any portion of the voice input toother devices, displaying, via the one or more display devices, thevisual indication includes displaying the visual indication with asecond set of one or more colors that is different from the first set ofone or more colors.

In accordance with some embodiments, a method is described. The methodis performed at a computer system, wherein the computer system is incommunication with a display generation component and one or more inputdevices. The method comprises: displaying, via the display generationcomponent, a first status that indicates that a plurality of accessorydevices of a first accessory type are in a first state, wherein theplurality of accessory devices of the first accessory type includes: afirst accessory device that is the first accessory type and is in thefirst state; and a second accessory device that is the first accessorytype and is in the first state, wherein the second accessory device isdifferent from the first accessory device; while displaying the firststatus indicator, detecting a gesture; and in response to detecting thegesture: in accordance with a determination that the gesture is a firsttype of gesture and is directed to the first status indicator, causingthe first accessory device and the second accessory device to change toa second state that is different from the first state; and in accordancewith a determination that the gesture is a second type of gesture thatis different from the first type of gesture and is directed to the firststatus indicator, displaying, via the display generation component, afirst accessory control user interface object corresponding to the firstaccessory device, wherein selection of the first accessory control userinterface object causes the first accessory device to change to thesecond state without causing the second accessory device to change tothe second state.

In accordance with some embodiments, a non-transitory computer-readablestorage medium is described. The non-transitory computer-readablestorage medium stores one or more programs configured to be executed byone or more processors of a computer system, wherein the computer systemis in communication with a display generation component and one or moreinput devices, the one or more programs including instructions for:displaying, via the display generation component, a first status thatindicates that a plurality of accessory devices of a first accessorytype are in a first state, wherein the plurality of accessory devices ofthe first accessory type includes: a first accessory device that is thefirst accessory type and is in the first state; and a second accessorydevice that is the first accessory type and is in the first state,wherein the second accessory device is different from the firstaccessory device; while displaying the first status indicator, detectinga gesture; and in response to detecting the gesture: in accordance witha determination that the gesture is a first type of gesture and isdirected to the first status indicator, causing the first accessorydevice and the second accessory device to change to a second state thatis different from the first state; and in accordance with adetermination that the gesture is a second type of gesture that isdifferent from the first type of gesture and is directed to the firststatus indicator, displaying, via the display generation component, afirst accessory control user interface object corresponding to the firstaccessory device, wherein selection of the first accessory control userinterface object causes the first accessory device to change to thesecond state without causing the second accessory device to change tothe second state.

In accordance with some embodiments, a transitory computer-readablestorage medium is described. The transitory computer-readable storagemedium stores one or more programs configured to be executed by one ormore processors of a computer system, wherein the computer system is incommunication with a display generation component and one or more inputdevices, the one or more programs including instructions for:displaying, via the display generation component, a first status thatindicates that a plurality of accessory devices of a first accessorytype are in a first state, wherein the plurality of accessory devices ofthe first accessory type includes: a first accessory device that is thefirst accessory type and is in the first state; and a second accessorydevice that is the first accessory type and is in the first state,wherein the second accessory device is different from the firstaccessory device; while displaying the first status indicator, detectinga gesture; and in response to detecting the gesture: in accordance witha determination that the gesture is a first type of gesture and isdirected to the first status indicator, causing the first accessorydevice and the second accessory device to change to a second state thatis different from the first state; and in accordance with adetermination that the gesture is a second type of gesture that isdifferent from the first type of gesture and is directed to the firststatus indicator, displaying, via the display generation component, afirst accessory control user interface object corresponding to the firstaccessory device, wherein selection of the first accessory control userinterface object causes the first accessory device to change to thesecond state without causing the second accessory device to change tothe second state.

In accordance with some embodiments, a computer system is described. Thecomputer system comprises: one or more processors, wherein the computersystem is in communication with a display generation component and oneor more input devices; and memory storing one or more programsconfigured to be executed by the one or more processors, the one or moreprograms including instructions for: displaying, via the displaygeneration component, a first status that indicates that a plurality ofaccessory devices of a first accessory type are in a first state,wherein the plurality of accessory devices of the first accessory typeincludes: a first accessory device that is the first accessory type andis in the first state; and a second accessory device that is the firstaccessory type and is in the first state, wherein the second accessorydevice is different from the first accessory device; while displayingthe first status indicator, detecting a gesture; and in response todetecting the gesture: in accordance with a determination that thegesture is a first type of gesture and is directed to the first statusindicator, causing the first accessory device and the second accessorydevice to change to a second state that is different from the firststate; and in accordance with a determination that the gesture is asecond type of gesture that is different from the first type of gestureand is directed to the first status indicator, displaying, via thedisplay generation component, a first accessory control user interfaceobject corresponding to the first accessory device, wherein selection ofthe first accessory control user interface object causes the firstaccessory device to change to the second state without causing thesecond accessory device to change to the second state.

In accordance with some embodiments, a computer system is described. Thecomputer system comprising: means for, displaying, via the displaygeneration component, a first status that indicates that a plurality ofaccessory devices of a first accessory type are in a first state,wherein the plurality of accessory devices of the first accessory typeincludes: a first accessory device that is the first accessory type andis in the first state; and a second accessory device that is the firstaccessory type and is in the first state, wherein the second accessorydevice is different from the first accessory device; means, whiledisplaying the first status indicator, for detecting a gesture; andmeans, responsive to detecting the gesture, for: in accordance with adetermination that the gesture is a first type of gesture and isdirected to the first status indicator, causing the first accessorydevice and the second accessory device to change to a second state thatis different from the first state; and in accordance with adetermination that the gesture is a second type of gesture that isdifferent from the first type of gesture and is directed to the firststatus indicator, displaying, via the display generation component, afirst accessory control user interface object corresponding to the firstaccessory device, wherein selection of the first accessory control userinterface object causes the first accessory device to change to thesecond state without causing the second accessory device to change tothe second state

In accordance with some embodiments, a method is described. The methodis performed at a computer system that is in communication with adisplay generation component and one or more inputs devices. The methodcomprises: displaying, via the display generation component, a mediauser interface that includes: at a first location, a firstrepresentation of a first media message, and at a second locationdifferent from the first location, a second representation of a secondmedia message; in response to displaying the media user interface andnot based on user input corresponding to the media user interface,playing the first media message; and in response to detecting completionof playing through the first media message: displaying, the secondrepresentation of the second media message without displaying, via thedisplay generation component, a representation corresponding to thefirst media message.

In accordance with some embodiments, a non-transitory computer-readablestorage medium is described. The non-transitory computer-readablestorage medium stores one or more programs configured to be executed byone or more processors of a computer system, wherein the computer systemis in communication with a display generation component and one or moreinput devices, the one or more programs including instructions for:displaying, via the display generation component, a media user interfacethat includes: at a first location, a first representation of a firstmedia message, and at a second location different from the firstlocation, a second representation of a second media message; in responseto displaying the media user interface and not based on user inputcorresponding to the media user interface, playing the first mediamessage; and in response to detecting completion of playing through thefirst media message: displaying, the second representation of the secondmedia message without displaying, via the display generation component,a representation corresponding to the first media message.

In accordance with some embodiments, a transitory computer-readablestorage medium is described. The transitory computer-readable storagemedium stores one or more programs configured to be executed by one ormore processors of a computer system, wherein the computer system is incommunication with a display generation component and one or more inputdevices, the one or more programs including instructions for:displaying, via the display generation component, a media user interfacethat includes: at a first location, a first representation of a firstmedia message, and at a second location different from the firstlocation, a second representation of a second media message; in responseto displaying the media user interface and not based on user inputcorresponding to the media user interface, playing the first mediamessage; and in response to detecting completion of playing through thefirst media message: displaying, the second representation of the secondmedia message without displaying, via the display generation component,a representation corresponding to the first media message.

In accordance with some embodiments, a computer system is described. Thecomputer system comprises: one or more processors, wherein the computersystem is in communication with a display generation component and oneor more input devices; and memory storing one or more programsconfigured to be executed by the one or more processors, the one or moreprograms including instructions for: displaying, via the displaygeneration component, a media user interface that includes: at a firstlocation, a first representation of a first media message, and at asecond location different from the first location, a secondrepresentation of a second media message; in response to displaying themedia user interface and not based on user input corresponding to themedia user interface, playing the first media message; and in responseto detecting completion of playing through the first media message:displaying, the second representation of the second media messagewithout displaying, via the display generation component, arepresentation corresponding to the first media message.

In accordance with some embodiments, a computer system is described. Thecomputer system is in communication with a display generation componentand one or more inputs devices. The computer system comprises: means fordisplaying, via the display generation component, a media user interfacethat includes: at a first location, a first representation of a firstmedia message, and at a second location different from the firstlocation, a second representation of a second media message; means,responsive to displaying the media user interface and not based on userinput corresponding to the media user interface, for playing the firstmedia message; and means, responsive to detecting completion of playingthrough the first media message, for: displaying, the secondrepresentation of the second media message without displaying, via thedisplay generation component, a representation corresponding to thefirst media message.

Executable instructions for performing these functions are, optionally,included in a non-transitory computer-readable storage medium or othercomputer program product configured for execution by one or moreprocessors. Executable instructions for performing these functions are,optionally, included in a transitory computer-readable storage medium orother computer program product configured for execution by one or moreprocessors.

Thus, devices are provided with faster, more efficient methods andinterfaces for managing audio messages, thereby increasing theeffectiveness, efficiency, and user satisfaction with such devices. Suchmethods and interfaces may complement or replace other methods formanaging audio messages.

DESCRIPTION OF THE FIGURES

For a better understanding of the various described embodiments,reference should be made to the Description of Embodiments below, inconjunction with the following drawings in which like reference numeralsrefer to corresponding parts throughout the figures.

FIG. 1A is a block diagram illustrating a portable multifunction devicewith a touch-sensitive display in accordance with some embodiments.

FIG. 1B is a block diagram illustrating exemplary components for eventhandling in accordance with some embodiments.

FIG. 2 illustrates a portable multifunction device having a touch screenin accordance with some embodiments.

FIG. 3 is a block diagram of an exemplary multifunction device with adisplay and a touch-sensitive surface in accordance with someembodiments.

FIG. 4A illustrates an exemplary user interface for a menu ofapplications on a portable multifunction device in accordance with someembodiments.

FIG. 4B illustrates an exemplary user interface for a multifunctiondevice with a touch-sensitive surface that is separate from the displayin accordance with some embodiments.

FIG. 5A illustrates a personal electronic device in accordance with someembodiments.

FIG. 5B is a block diagram illustrating a personal electronic device inaccordance with some embodiments.

FIGS. 5C-5D illustrate exemplary components of a personal electronicdevice having a touch-sensitive display and intensity sensors inaccordance with some embodiments.

FIGS. 5E-5H illustrate exemplary components and user interfaces of apersonal electronic device in accordance with some embodiments.

FIG. 5I illustrates an electronic device in accordance with someembodiments.

FIG. 5J is a block diagram illustrating an electronic device inaccordance with some embodiments.

FIG. 6 is a diagram illustrating a physical structure and an exemplaryset of devices in accordance with some embodiments.

FIGS. 7A-7AH illustrate exemplary user interfaces for sending an audiomessage in accordance with some embodiments.

FIGS. 8A-8B are a flow diagram illustrating a method for sending anaudio message in accordance with some embodiments.

FIGS. 9A-9X illustrate exemplary user interfaces for displayingnotifications in accordance with some embodiments.

FIG. 10 is a flow diagram illustrating a method for displayingnotifications in accordance with some embodiments.

FIGS. 11A-11R illustrate exemplary user interfaces for displaying visualindications, in accordance with some embodiments.

FIG. 12 is a flow diagram illustrating a method for displaying visualindications in accordance with some embodiments.

FIGS. 13A-13N illustrate exemplary user interfaces for managing externaldevices in accordance with some embodiments.

FIG. 14 is a flow diagram illustrating a method for managing externaldevices in accordance with some embodiments.

FIGS. 15A-15F illustrate exemplary user interfaces for playing backaudio messages in accordance with some embodiments.

FIG. 16 is a flow diagram illustrating a method for playing back audiomessages in accordance with some embodiments.

DESCRIPTION OF EMBODIMENTS

The following description sets forth exemplary methods, parameters, andthe like. It should be recognized, however, that such description is notintended as a limitation on the scope of the present disclosure but isinstead provided as a description of exemplary embodiments.

There is a need for electronic devices that provide efficient methodsand interfaces for managing audio messages. The techniques for managingaudio messages increase the efficiency of the devices (e.g., reduceunnecessary display), increase information provided about externaldevices (e.g., displaying the source of audio messages), and increaseprivacy (by providing visual feedback to the user of the state of thedevice about audio input received). Such techniques can reduce thecognitive burden on a user who uses audio messages, thereby enhancingproductivity. Further, such techniques can reduce processor and batterypower otherwise wasted on redundant user inputs.

Below, FIGS. 1A-1B, 2, 3, 4A-4B, and 5A-5J provide a description ofexemplary devices for performing the techniques for sending andreceiving audio messages. FIG. 6 is a diagram illustrating a physicalstructure and an exemplary set of devices for performing the techniquesfor sending and receiving audio messages, in accordance with someembodiments.

FIGS. 7A-7AH illustrate exemplary user interfaces for sending an audiomessage, in accordance with some embodiments. FIGS. 8A-8B are a flowdiagram illustrating a method for sending an audio message, inaccordance with some embodiments. The user interfaces in FIGS. 7A-7AHare used to illustrate the processes described below, including theprocesses in FIGS. 8A-8B.

FIGS. 9A-9X illustrate exemplary user interfaces for displayingnotifications, in accordance with some embodiments. FIG. 10 is a flowdiagram illustrating a method for displaying notifications, inaccordance with some embodiments. The user interfaces in FIGS. 9A-9X areused to illustrate the processes described below, including theprocesses in FIG. 10.

FIGS. 11A-11R illustrate exemplary user interfaces for displaying visualindications. FIG. 12 is a flow diagram illustrating a method fordisplaying visual indications. The user interfaces in FIGS. 11A-11R areused to illustrate the processes described below, including theprocesses in FIG. 12.

FIGS. 13A-13N illustrate exemplary user interfaces for managing statusesof devices. FIG. 14 is a flow diagram illustrating a method for managingstatuses of devices. The user interfaces in FIGS. 13A-13N are used toillustrate the processes described below, including the processes inFIG. 14.

FIGS. 15A-15F illustrate exemplary user interfaces for playing backaudio messages in accordance with some embodiments. FIG. 16 is a flowdiagram illustrating a method for playing back audio messages inaccordance with some embodiments. The user interfaces in FIGS. 15A-15Fare used to illustrate the processes described below, including theprocesses in FIG. 16.

Although the following description uses terms “first,” “second,” etc. todescribe various elements, these elements should not be limited by theterms. These terms are only used to distinguish one element fromanother. For example, a first touch could be termed a second touch, and,similarly, a second touch could be termed a first touch, withoutdeparting from the scope of the various described embodiments. The firsttouch and the second touch are both touches, but they are not the sametouch.

The terminology used in the description of the various describedembodiments herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used in thedescription of the various described embodiments and the appendedclaims, the singular forms “a,” “an,” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. It will also be understood that the term “and/or” as usedherein refers to and encompasses any and all possible combinations ofone or more of the associated listed items. It will be furtherunderstood that the terms “includes,” “including,” “comprises,” and/or“comprising,” when used in this specification, specify the presence ofstated features, integers, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, integers, steps, operations, elements, components,and/or groups thereof.

The term “if” is, optionally, construed to mean “when” or “upon” or “inresponse to determining” or “in response to detecting,” depending on thecontext. Similarly, the phrase “if it is determined” or “if [a statedcondition or event] is detected” is, optionally, construed to mean “upondetermining” or “in response to determining” or “upon detecting [thestated condition or event]” or “in response to detecting [the statedcondition or event],” depending on the context.

Embodiments of electronic devices, user interfaces for such devices, andassociated processes for using such devices are described. In someembodiments, the device is a portable communications device, such as amobile telephone, that also contains other functions, such as PDA and/ormusic player functions. Exemplary embodiments of portable multifunctiondevices include, without limitation, the iPhone®, iPod Touch®, and iPad®devices from Apple Inc. of Cupertino, Calif. Other portable electronicdevices, such as laptops or tablet computers with touch-sensitivesurfaces (e.g., touch screen displays and/or touchpads), are,optionally, used. It should also be understood that, in someembodiments, the device is not a portable communications device, but isa desktop computer with a touch-sensitive surface (e.g., a touch screendisplay and/or a touchpad).

In the discussion that follows, an electronic device that includes adisplay and a touch-sensitive surface is described. It should beunderstood, however, that the electronic device optionally includes oneor more other physical user-interface devices, such as a physicalkeyboard, a mouse, and/or a joystick.

The device typically supports a variety of applications, such as one ormore of the following: a drawing application, a presentationapplication, a word processing application, a website creationapplication, a disk authoring application, a spreadsheet application, agaming application, a telephone application, a video conferencingapplication, an e-mail application, an instant messaging application, aworkout support application, a photo management application, a digitalcamera application, a digital video camera application, a web browsingapplication, a digital music player application, and/or a digital videoplayer application.

The various applications that are executed on the device optionally useat least one common physical user-interface device, such as thetouch-sensitive surface. One or more functions of the touch-sensitivesurface as well as corresponding information displayed on the deviceare, optionally, adjusted and/or varied from one application to the nextand/or within a respective application. In this way, a common physicalarchitecture (such as the touch-sensitive surface) of the deviceoptionally supports the variety of applications with user interfacesthat are intuitive and transparent to the user.

Attention is now directed toward embodiments of portable devices withtouch-sensitive displays. FIG. 1A is a block diagram illustratingportable multifunction device 100 with touch-sensitive display system112 in accordance with some embodiments. Touch-sensitive display 112 issometimes called a “touch screen” for convenience and is sometimes knownas or called a “touch-sensitive display system.” Device 100 includesmemory 102 (which optionally includes one or more computer-readablestorage mediums), memory controller 122, one or more processing units(CPUs) 120, peripherals interface 118, RF circuitry 108, audio circuitry110, speaker 111, microphone 113, input/output (I/O) subsystem 106,other input control devices 116, and external port 124. Device 100optionally includes one or more optical sensors 164. Device 100optionally includes one or more contact intensity sensors 165 fordetecting intensity of contacts on device 100 (e.g., a touch-sensitivesurface such as touch-sensitive display system 112 of device 100).Device 100 optionally includes one or more tactile output generators 167for generating tactile outputs on device 100 (e.g., generating tactileoutputs on a touch-sensitive surface such as touch-sensitive displaysystem 112 of device 100 or touchpad 355 of device 300). Thesecomponents optionally communicate over one or more communication busesor signal lines 103.

As used in the specification and claims, the term “intensity” of acontact on a touch-sensitive surface refers to the force or pressure(force per unit area) of a contact (e.g., a finger contact) on thetouch-sensitive surface, or to a substitute (proxy) for the force orpressure of a contact on the touch-sensitive surface. The intensity of acontact has a range of values that includes at least four distinctvalues and more typically includes hundreds of distinct values (e.g., atleast 256). Intensity of a contact is, optionally, determined (ormeasured) using various approaches and various sensors or combinationsof sensors. For example, one or more force sensors underneath oradjacent to the touch-sensitive surface are, optionally, used to measureforce at various points on the touch-sensitive surface. In someimplementations, force measurements from multiple force sensors arecombined (e.g., a weighted average) to determine an estimated force of acontact. Similarly, a pressure-sensitive tip of a stylus is, optionally,used to determine a pressure of the stylus on the touch-sensitivesurface. Alternatively, the size of the contact area detected on thetouch-sensitive surface and/or changes thereto, the capacitance of thetouch-sensitive surface proximate to the contact and/or changes thereto,and/or the resistance of the touch-sensitive surface proximate to thecontact and/or changes thereto are, optionally, used as a substitute forthe force or pressure of the contact on the touch-sensitive surface. Insome implementations, the substitute measurements for contact force orpressure are used directly to determine whether an intensity thresholdhas been exceeded (e.g., the intensity threshold is described in unitscorresponding to the substitute measurements). In some implementations,the substitute measurements for contact force or pressure are convertedto an estimated force or pressure, and the estimated force or pressureis used to determine whether an intensity threshold has been exceeded(e.g., the intensity threshold is a pressure threshold measured in unitsof pressure). Using the intensity of a contact as an attribute of a userinput allows for user access to additional device functionality that mayotherwise not be accessible by the user on a reduced-size device withlimited real estate for displaying affordances (e.g., on atouch-sensitive display) and/or receiving user input (e.g., via atouch-sensitive display, a touch-sensitive surface, or aphysical/mechanical control such as a knob or a button).

As used in the specification and claims, the term “tactile output”refers to physical displacement of a device relative to a previousposition of the device, physical displacement of a component (e.g., atouch-sensitive surface) of a device relative to another component(e.g., housing) of the device, or displacement of the component relativeto a center of mass of the device that will be detected by a user withthe user's sense of touch. For example, in situations where the deviceor the component of the device is in contact with a surface of a userthat is sensitive to touch (e.g., a finger, palm, or other part of auser's hand), the tactile output generated by the physical displacementwill be interpreted by the user as a tactile sensation corresponding toa perceived change in physical characteristics of the device or thecomponent of the device. For example, movement of a touch-sensitivesurface (e.g., a touch-sensitive display or trackpad) is, optionally,interpreted by the user as a “down click” or “up click” of a physicalactuator button. In some cases, a user will feel a tactile sensationsuch as an “down click” or “up click” even when there is no movement ofa physical actuator button associated with the touch-sensitive surfacethat is physically pressed (e.g., displaced) by the user's movements. Asanother example, movement of the touch-sensitive surface is, optionally,interpreted or sensed by the user as “roughness” of the touch-sensitivesurface, even when there is no change in smoothness of thetouch-sensitive surface. While such interpretations of touch by a userwill be subject to the individualized sensory perceptions of the user,there are many sensory perceptions of touch that are common to a largemajority of users. Thus, when a tactile output is described ascorresponding to a particular sensory perception of a user (e.g., an “upclick,” a “down click,” “roughness”), unless otherwise stated, thegenerated tactile output corresponds to physical displacement of thedevice or a component thereof that will generate the described sensoryperception for a typical (or average) user.

It should be appreciated that device 100 is only one example of aportable multifunction device, and that device 100 optionally has moreor fewer components than shown, optionally combines two or morecomponents, or optionally has a different configuration or arrangementof the components. The various components shown in FIG. 1A areimplemented in hardware, software, or a combination of both hardware andsoftware, including one or more signal processing and/orapplication-specific integrated circuits.

Memory 102 optionally includes high-speed random access memory andoptionally also includes non-volatile memory, such as one or moremagnetic disk storage devices, flash memory devices, or othernon-volatile solid-state memory devices. Memory controller 122optionally controls access to memory 102 by other components of device100.

Peripherals interface 118 can be used to couple input and outputperipherals of the device to CPU 120 and memory 102. The one or moreprocessors 120 run or execute various software programs and/or sets ofinstructions stored in memory 102 to perform various functions fordevice 100 and to process data. In some embodiments, peripheralsinterface 118, CPU 120, and memory controller 122 are, optionally,implemented on a single chip, such as chip 104. In some otherembodiments, they are, optionally, implemented on separate chips.

RF (radio frequency) circuitry 108 receives and sends RF signals, alsocalled electromagnetic signals. RF circuitry 108 converts electricalsignals to/from electromagnetic signals and communicates withcommunications networks and other communications devices via theelectromagnetic signals. RF circuitry 108 optionally includes well-knowncircuitry for performing these functions, including but not limited toan antenna system, an RF transceiver, one or more amplifiers, a tuner,one or more oscillators, a digital signal processor, a CODEC chipset, asubscriber identity module (SIM) card, memory, and so forth. RFcircuitry 108 optionally communicates with networks, such as theInternet, also referred to as the World Wide Web (WWW), an intranetand/or a wireless network, such as a cellular telephone network, awireless local area network (LAN) and/or a metropolitan area network(MAN), and other devices by wireless communication. The RF circuitry 108optionally includes well-known circuitry for detecting near fieldcommunication (NFC) fields, such as by a short-range communicationradio. The wireless communication optionally uses any of a plurality ofcommunications standards, protocols, and technologies, including but notlimited to Global System for Mobile Communications (GSM), Enhanced DataGSM Environment (EDGE), high-speed downlink packet access (HSDPA),high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO),HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), long term evolution (LTE), nearfield communication (NFC), wideband code division multiple access(W-CDMA), code division multiple access (CDMA), time division multipleaccess (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), Wireless Fidelity(Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n,and/or IEEE 802.11ac), voice over Internet Protocol (VoIP), Wi-MAX, aprotocol for e-mail (e.g., Internet message access protocol (IMAP)and/or post office protocol (POP)), instant messaging (e.g., extensiblemessaging and presence protocol (XMPP), Session Initiation Protocol forInstant Messaging and Presence Leveraging Extensions (SIMPLE), InstantMessaging and Presence Service (IMPS)), and/or Short Message Service(SMS), or any other suitable communication protocol, includingcommunication protocols not yet developed as of the filing date of thisdocument.

Audio circuitry 110, speaker 111, and microphone 113 provide an audiointerface between a user and device 100. Audio circuitry 110 receivesaudio data from peripherals interface 118, converts the audio data to anelectrical signal, and transmits the electrical signal to speaker 111.Speaker 111 converts the electrical signal to human-audible sound waves.Audio circuitry 110 also receives electrical signals converted bymicrophone 113 from sound waves. Audio circuitry 110 converts theelectrical signal to audio data and transmits the audio data toperipherals interface 118 for processing. Audio data is, optionally,retrieved from and/or transmitted to memory 102 and/or RF circuitry 108by peripherals interface 118. In some embodiments, audio circuitry 110also includes a headset jack (e.g., 212, FIG. 2). The headset jackprovides an interface between audio circuitry 110 and removable audioinput/output peripherals, such as output-only headphones or a headsetwith both output (e.g., a headphone for one or both ears) and input(e.g., a microphone).

I/O subsystem 106 couples input/output peripherals on device 100, suchas touch screen 112 and other input control devices 116, to peripheralsinterface 118. I/O subsystem 106 optionally includes display controller156, optical sensor controller 158, depth camera controller 169,intensity sensor controller 159, haptic feedback controller 161, and oneor more input controllers 160 for other input or control devices. Theone or more input controllers 160 receive/send electrical signalsfrom/to other input control devices 116. The other input control devices116 optionally include physical buttons (e.g., push buttons, rockerbuttons, etc.), dials, slider switches, joysticks, click wheels, and soforth. In some alternate embodiments, input controller(s) 160 are,optionally, coupled to any (or none) of the following: a keyboard, aninfrared port, a USB port, and a pointer device such as a mouse. The oneor more buttons (e.g., 208, FIG. 2) optionally include an up/down buttonfor volume control of speaker 111 and/or microphone 113. The one or morebuttons optionally include a push button (e.g., 206, FIG. 2).

A quick press of the push button optionally disengages a lock of touchscreen 112 or optionally begins a process that uses gestures on thetouch screen to unlock the device, as described in U.S. patentapplication Ser. No. 11/322,549, “Unlocking a Device by PerformingGestures on an Unlock Image,” filed Dec. 23, 2005, U.S. Pat. No.7,657,849, which is hereby incorporated by reference in its entirety. Alonger press of the push button (e.g., 206) optionally turns power todevice 100 on or off. The functionality of one or more of the buttonsare, optionally, user-customizable. Touch screen 112 is used toimplement virtual or soft buttons and one or more soft keyboards.

Touch-sensitive display 112 provides an input interface and an outputinterface between the device and a user. Display controller 156 receivesand/or sends electrical signals from/to touch screen 112. Touch screen112 displays visual output to the user. The visual output optionallyincludes graphics, text, icons, video, and any combination thereof(collectively termed “graphics”). In some embodiments, some or all ofthe visual output optionally corresponds to user-interface objects.

Touch screen 112 has a touch-sensitive surface, sensor, or set ofsensors that accepts input from the user based on haptic and/or tactilecontact. Touch screen 112 and display controller 156 (along with anyassociated modules and/or sets of instructions in memory 102) detectcontact (and any movement or breaking of the contact) on touch screen112 and convert the detected contact into interaction withuser-interface objects (e.g., one or more soft keys, icons, web pages,or images) that are displayed on touch screen 112. In an exemplaryembodiment, a point of contact between touch screen 112 and the usercorresponds to a finger of the user.

Touch screen 112 optionally uses LCD (liquid crystal display)technology, LPD (light emitting polymer display) technology, or LED(light emitting diode) technology, although other display technologiesare used in other embodiments. Touch screen 112 and display controller156 optionally detect contact and any movement or breaking thereof usingany of a plurality of touch sensing technologies now known or laterdeveloped, including but not limited to capacitive, resistive, infrared,and surface acoustic wave technologies, as well as other proximitysensor arrays or other elements for determining one or more points ofcontact with touch screen 112. In an exemplary embodiment, projectedmutual capacitance sensing technology is used, such as that found in theiPhone® and iPod Touch® from Apple Inc. of Cupertino, Calif.

A touch-sensitive display in some embodiments of touch screen 112 is,optionally, analogous to the multi-touch sensitive touchpads describedin the following U.S. Pat. No. 6,323,846 (Westerman et al.), U.S. Pat.No. 6,570,557 (Westerman et al.), and/or U.S. Pat. No. 6,677,932(Westerman), and/or U.S. Patent Publication 2002/0015024A1, each ofwhich is hereby incorporated by reference in its entirety. However,touch screen 112 displays visual output from device 100, whereastouch-sensitive touchpads do not provide visual output.

A touch-sensitive display in some embodiments of touch screen 112 isdescribed in the following applications: (1) U.S. patent applicationSer. No. 11/381,313, “Multipoint Touch Surface Controller,” filed May 2,2006; (2) U.S. patent application Ser. No. 10/840,862, “MultipointTouchscreen,” filed May 6, 2004; (3) U.S. patent application Ser. No.10/903,964, “Gestures For Touch Sensitive Input Devices,” filed Jul. 30,2004; (4) U.S. patent application Ser. No. 11/048,264, “Gestures ForTouch Sensitive Input Devices,” filed Jan. 31, 2005; (5) U.S. patentapplication Ser. No. 11/038,590, “Mode-Based Graphical User InterfacesFor Touch Sensitive Input Devices,” filed Jan. 18, 2005; (6) U.S. patentapplication Ser. No. 11/228,758, “Virtual Input Device Placement On ATouch Screen User Interface,” filed Sep. 16, 2005; (7) U.S. patentapplication Ser. No. 11/228,700, “Operation Of A Computer With A TouchScreen Interface,” filed Sep. 16, 2005; (8) U.S. patent application Ser.No. 11/228,737, “Activating Virtual Keys Of A Touch-Screen VirtualKeyboard,” filed Sep. 16, 2005; and (9) U.S. patent application Ser. No.11/367,749, “Multi-Functional Hand-Held Device,” filed Mar. 3, 2006. Allof these applications are incorporated by reference herein in theirentirety.

Touch screen 112 optionally has a video resolution in excess of 100 dpi.In some embodiments, the touch screen has a video resolution ofapproximately 160 dpi. The user optionally makes contact with touchscreen 112 using any suitable object or appendage, such as a stylus, afinger, and so forth. In some embodiments, the user interface isdesigned to work primarily with finger-based contacts and gestures,which can be less precise than stylus-based input due to the larger areaof contact of a finger on the touch screen. In some embodiments, thedevice translates the rough finger-based input into a precisepointer/cursor position or command for performing the actions desired bythe user.

In some embodiments, in addition to the touch screen, device 100optionally includes a touchpad for activating or deactivating particularfunctions. In some embodiments, the touchpad is a touch-sensitive areaof the device that, unlike the touch screen, does not display visualoutput. The touchpad is, optionally, a touch-sensitive surface that isseparate from touch screen 112 or an extension of the touch-sensitivesurface formed by the touch screen.

Device 100 also includes power system 162 for powering the variouscomponents. Power system 162 optionally includes a power managementsystem, one or more power sources (e.g., battery, alternating current(AC)), a recharging system, a power failure detection circuit, a powerconverter or inverter, a power status indicator (e.g., a light-emittingdiode (LED)) and any other components associated with the generation,management and distribution of power in portable devices.

Device 100 optionally also includes one or more optical sensors 164.FIG. 1A shows an optical sensor coupled to optical sensor controller 158in I/O subsystem 106. Optical sensor 164 optionally includescharge-coupled device (CCD) or complementary metal-oxide semiconductor(CMOS) phototransistors. Optical sensor 164 receives light from theenvironment, projected through one or more lenses, and converts thelight to data representing an image. In conjunction with imaging module143 (also called a camera module), optical sensor 164 optionallycaptures still images or video. In some embodiments, an optical sensoris located on the back of device 100, opposite touch screen display 112on the front of the device so that the touch screen display is enabledfor use as a viewfinder for still and/or video image acquisition. Insome embodiments, an optical sensor is located on the front of thedevice so that the user's image is, optionally, obtained for videoconferencing while the user views the other video conferenceparticipants on the touch screen display. In some embodiments, theposition of optical sensor 164 can be changed by the user (e.g., byrotating the lens and the sensor in the device housing) so that a singleoptical sensor 164 is used along with the touch screen display for bothvideo conferencing and still and/or video image acquisition.

Device 100 optionally also includes one or more depth camera sensors175. FIG. 1A shows a depth camera sensor coupled to depth cameracontroller 169 in I/O subsystem 106. Depth camera sensor 175 receivesdata from the environment to create a three dimensional model of anobject (e.g., a face) within a scene from a viewpoint (e.g., a depthcamera sensor). In some embodiments, in conjunction with imaging module143 (also called a camera module), depth camera sensor 175 is optionallyused to determine a depth map of different portions of an image capturedby the imaging module 143. In some embodiments, a depth camera sensor islocated on the front of device 100 so that the user's image with depthinformation is, optionally, obtained for video conferencing while theuser views the other video conference participants on the touch screendisplay and to capture selfies with depth map data. In some embodiments,the depth camera sensor 175 is located on the back of device, or on theback and the front of the device 100. In some embodiments, the positionof depth camera sensor 175 can be changed by the user (e.g., by rotatingthe lens and the sensor in the device housing) so that a depth camerasensor 175 is used along with the touch screen display for both videoconferencing and still and/or video image acquisition.

Device 100 optionally also includes one or more contact intensitysensors 165. FIG. 1A shows a contact intensity sensor coupled tointensity sensor controller 159 in I/O subsystem 106. Contact intensitysensor 165 optionally includes one or more piezoresistive strain gauges,capacitive force sensors, electric force sensors, piezoelectric forcesensors, optical force sensors, capacitive touch-sensitive surfaces, orother intensity sensors (e.g., sensors used to measure the force (orpressure) of a contact on a touch-sensitive surface). Contact intensitysensor 165 receives contact intensity information (e.g., pressureinformation or a proxy for pressure information) from the environment.In some embodiments, at least one contact intensity sensor is collocatedwith, or proximate to, a touch-sensitive surface (e.g., touch-sensitivedisplay system 112). In some embodiments, at least one contact intensitysensor is located on the back of device 100, opposite touch screendisplay 112, which is located on the front of device 100.

Device 100 optionally also includes one or more proximity sensors 166.FIG. 1A shows proximity sensor 166 coupled to peripherals interface 118.Alternately, proximity sensor 166 is, optionally, coupled to inputcontroller 160 in I/O subsystem 106. Proximity sensor 166 optionallyperforms as described in U.S. patent application Ser. No. 11/241,839,“Proximity Detector In Handheld Device”; Ser. No. 11/240,788, “ProximityDetector In Handheld Device”; Ser. No. 11/620,702, “Using Ambient LightSensor To Augment Proximity Sensor Output”; Ser. No. 11/586,862,“Automated Response To And Sensing Of User Activity In PortableDevices”; and Ser. No. 11/638,251, “Methods And Systems For AutomaticConfiguration Of Peripherals,” which are hereby incorporated byreference in their entirety. In some embodiments, the proximity sensorturns off and disables touch screen 112 when the multifunction device isplaced near the user's ear (e.g., when the user is making a phone call).

Device 100 optionally also includes one or more tactile outputgenerators 167. FIG. 1A shows a tactile output generator coupled tohaptic feedback controller 161 in I/O subsystem 106. Tactile outputgenerator 167 optionally includes one or more electroacoustic devicessuch as speakers or other audio components and/or electromechanicaldevices that convert energy into linear motion such as a motor,solenoid, electroactive polymer, piezoelectric actuator, electrostaticactuator, or other tactile output generating component (e.g., acomponent that converts electrical signals into tactile outputs on thedevice). Contact intensity sensor 165 receives tactile feedbackgeneration instructions from haptic feedback module 133 and generatestactile outputs on device 100 that are capable of being sensed by a userof device 100. In some embodiments, at least one tactile outputgenerator is collocated with, or proximate to, a touch-sensitive surface(e.g., touch-sensitive display system 112) and, optionally, generates atactile output by moving the touch-sensitive surface vertically (e.g.,in/out of a surface of device 100) or laterally (e.g., back and forth inthe same plane as a surface of device 100). In some embodiments, atleast one tactile output generator sensor is located on the back ofdevice 100, opposite touch screen display 112, which is located on thefront of device 100.

Device 100 optionally also includes one or more accelerometers 168. FIG.1A shows accelerometer 168 coupled to peripherals interface 118.Alternately, accelerometer 168 is, optionally, coupled to an inputcontroller 160 in I/O subsystem 106. Accelerometer 168 optionallyperforms as described in U.S. Patent Publication No. 20050190059,“Acceleration-based Theft Detection System for Portable ElectronicDevices,” and U.S. Patent Publication No. 20060017692, “Methods AndApparatuses For Operating A Portable Device Based On An Accelerometer,”both of which are incorporated by reference herein in their entirety. Insome embodiments, information is displayed on the touch screen displayin a portrait view or a landscape view based on an analysis of datareceived from the one or more accelerometers. Device 100 optionallyincludes, in addition to accelerometer(s) 168, a magnetometer and a GPS(or GLONASS or other global navigation system) receiver for obtaininginformation concerning the location and orientation (e.g., portrait orlandscape) of device 100.

In some embodiments, the software components stored in memory 102include operating system 126, communication module (or set ofinstructions) 128, contact/motion module (or set of instructions) 130,graphics module (or set of instructions) 132, text input module (or setof instructions) 134, Global Positioning System (GPS) module (or set ofinstructions) 135, and applications (or sets of instructions) 136.Furthermore, in some embodiments, memory 102 (FIG. 1A) or 370 (FIG. 3)stores device/global internal state 157, as shown in FIGS. 1A and 3.Device/global internal state 157 includes one or more of: activeapplication state, indicating which applications, if any, are currentlyactive; display state, indicating what applications, views or otherinformation occupy various regions of touch screen display 112; sensorstate, including information obtained from the device's various sensorsand input control devices 116; and location information concerning thedevice's location and/or attitude.

Operating system 126 (e.g., Darwin, RTXC, LINUX, UNIX, OS X, iOS,WINDOWS, or an embedded operating system such as VxWorks) includesvarious software components and/or drivers for controlling and managinggeneral system tasks (e.g., memory management, storage device control,power management, etc.) and facilitates communication between varioushardware and software components.

Communication module 128 facilitates communication with other devicesover one or more external ports 124 and also includes various softwarecomponents for handling data received by RF circuitry 108 and/orexternal port 124. External port 124 (e.g., Universal Serial Bus (USB),FIREWIRE, etc.) is adapted for coupling directly to other devices orindirectly over a network (e.g., the Internet, wireless LAN, etc.). Insome embodiments, the external port is a multi-pin (e.g., 30-pin)connector that is the same as, or similar to and/or compatible with, the30-pin connector used on iPod® (trademark of Apple Inc.) devices.

Contact/motion module 130 optionally detects contact with touch screen112 (in conjunction with display controller 156) and othertouch-sensitive devices (e.g., a touchpad or physical click wheel).Contact/motion module 130 includes various software components forperforming various operations related to detection of contact, such asdetermining if contact has occurred (e.g., detecting a finger-downevent), determining an intensity of the contact (e.g., the force orpressure of the contact or a substitute for the force or pressure of thecontact), determining if there is movement of the contact and trackingthe movement across the touch-sensitive surface (e.g., detecting one ormore finger-dragging events), and determining if the contact has ceased(e.g., detecting a finger-up event or a break in contact).Contact/motion module 130 receives contact data from the touch-sensitivesurface. Determining movement of the point of contact, which isrepresented by a series of contact data, optionally includes determiningspeed (magnitude), velocity (magnitude and direction), and/or anacceleration (a change in magnitude and/or direction) of the point ofcontact. These operations are, optionally, applied to single contacts(e.g., one finger contacts) or to multiple simultaneous contacts (e.g.,“multitouch”/multiple finger contacts). In some embodiments,contact/motion module 130 and display controller 156 detect contact on atouchpad.

In some embodiments, contact/motion module 130 uses a set of one or moreintensity thresholds to determine whether an operation has beenperformed by a user (e.g., to determine whether a user has “clicked” onan icon). In some embodiments, at least a subset of the intensitythresholds are determined in accordance with software parameters (e.g.,the intensity thresholds are not determined by the activation thresholdsof particular physical actuators and can be adjusted without changingthe physical hardware of device 100). For example, a mouse “click”threshold of a trackpad or touch screen display can be set to any of alarge range of predefined threshold values without changing the trackpador touch screen display hardware. Additionally, in some implementations,a user of the device is provided with software settings for adjustingone or more of the set of intensity thresholds (e.g., by adjustingindividual intensity thresholds and/or by adjusting a plurality ofintensity thresholds at once with a system-level click “intensity”parameter).

Contact/motion module 130 optionally detects a gesture input by a user.Different gestures on the touch-sensitive surface have different contactpatterns (e.g., different motions, timings, and/or intensities ofdetected contacts). Thus, a gesture is, optionally, detected bydetecting a particular contact pattern. For example, detecting a fingertap gesture includes detecting a finger-down event followed by detectinga finger-up (liftoff) event at the same position (or substantially thesame position) as the finger-down event (e.g., at the position of anicon). As another example, detecting a finger swipe gesture on thetouch-sensitive surface includes detecting a finger-down event followedby detecting one or more finger-dragging events, and subsequentlyfollowed by detecting a finger-up (liftoff) event.

Graphics module 132 includes various known software components forrendering and displaying graphics on touch screen 112 or other display,including components for changing the visual impact (e.g., brightness,transparency, saturation, contrast, or other visual property) ofgraphics that are displayed. As used herein, the term “graphics”includes any object that can be displayed to a user, including, withoutlimitation, text, web pages, icons (such as user-interface objectsincluding soft keys), digital images, videos, animations, and the like.

In some embodiments, graphics module 132 stores data representinggraphics to be used. Each graphic is, optionally, assigned acorresponding code. Graphics module 132 receives, from applicationsetc., one or more codes specifying graphics to be displayed along with,if necessary, coordinate data and other graphic property data, and thengenerates screen image data to output to display controller 156.

Haptic feedback module 133 includes various software components forgenerating instructions used by tactile output generator(s) 167 toproduce tactile outputs at one or more locations on device 100 inresponse to user interactions with device 100.

Text input module 134, which is, optionally, a component of graphicsmodule 132, provides soft keyboards for entering text in variousapplications (e.g., contacts 137, e-mail 140, IM 141, browser 147, andany other application that needs text input).

GPS module 135 determines the location of the device and provides thisinformation for use in various applications (e.g., to telephone 138 foruse in location-based dialing; to camera 143 as picture/video metadata;and to applications that provide location-based services such as weatherwidgets, local yellow page widgets, and map/navigation widgets).

Applications 136 optionally include the following modules (or sets ofinstructions), or a subset or superset thereof:

-   -   Contacts module 137 (sometimes called an address book or contact        list);    -   Telephone module 138;    -   Video conference module 139;    -   E-mail client module 140;    -   Instant messaging (IM) module 141;    -   Workout support module 142;    -   Camera module 143 for still and/or video images;    -   Image management module 144;    -   Video player module;    -   Music player module;    -   Browser module 147;    -   Calendar module 148;    -   Widget modules 149, which optionally include one or more of:        weather widget 149-1, stocks widget 149-2, calculator widget        149-3, alarm clock widget 149-4, dictionary widget 149-5, and        other widgets obtained by the user, as well as user-created        widgets 149-6;    -   Widget creator module 150 for making user-created widgets 149-6;    -   Search module 151;    -   Video and music player module 152, which merges video player        module and music player module;    -   Notes module 153;    -   Map module 154; and/or    -   Online video module 155.

Examples of other applications 136 that are, optionally, stored inmemory 102 include other word processing applications, other imageediting applications, drawing applications, presentation applications,JAVA-enabled applications, encryption, digital rights management, voicerecognition, and voice replication.

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, and text input module134, contacts module 137 are, optionally, used to manage an address bookor contact list (e.g., stored in application internal state 192 ofcontacts module 137 in memory 102 or memory 370), including: addingname(s) to the address book; deleting name(s) from the address book;associating telephone number(s), e-mail address(es), physicaladdress(es) or other information with a name; associating an image witha name; categorizing and sorting names; providing telephone numbers ore-mail addresses to initiate and/or facilitate communications bytelephone 138, video conference module 139, e-mail 140, or IM 141; andso forth.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111,microphone 113, touch screen 112, display controller 156, contact/motionmodule 130, graphics module 132, and text input module 134, telephonemodule 138 are optionally, used to enter a sequence of characterscorresponding to a telephone number, access one or more telephonenumbers in contacts module 137, modify a telephone number that has beenentered, dial a respective telephone number, conduct a conversation, anddisconnect or hang up when the conversation is completed. As notedabove, the wireless communication optionally uses any of a plurality ofcommunications standards, protocols, and technologies.

In conjunction with RF circuitry 108, audio circuitry 110, speaker 111,microphone 113, touch screen 112, display controller 156, optical sensor164, optical sensor controller 158, contact/motion module 130, graphicsmodule 132, text input module 134, contacts module 137, and telephonemodule 138, video conference module 139 includes executable instructionsto initiate, conduct, and terminate a video conference between a userand one or more other participants in accordance with user instructions.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, and textinput module 134, e-mail client module 140 includes executableinstructions to create, send, receive, and manage e-mail in response touser instructions. In conjunction with image management module 144,e-mail client module 140 makes it very easy to create and send e-mailswith still or video images taken with camera module 143.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, and textinput module 134, the instant messaging module 141 includes executableinstructions to enter a sequence of characters corresponding to aninstant message, to modify previously entered characters, to transmit arespective instant message (for example, using a Short Message Service(SMS) or Multimedia Message Service (MMS) protocol for telephony-basedinstant messages or using XMPP, SIMPLE, or IMPS for Internet-basedinstant messages), to receive instant messages, and to view receivedinstant messages. In some embodiments, transmitted and/or receivedinstant messages optionally include graphics, photos, audio files, videofiles and/or other attachments as are supported in an MMS and/or anEnhanced Messaging Service (EMS). As used herein, “instant messaging”refers to both telephony-based messages (e.g., messages sent using SMSor MMS) and Internet-based messages (e.g., messages sent using XMPP,SIMPLE, or IMPS).

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, textinput module 134, GPS module 135, map module 154, and music playermodule, workout support module 142 includes executable instructions tocreate workouts (e.g., with time, distance, and/or calorie burninggoals); communicate with workout sensors (sports devices); receiveworkout sensor data; calibrate sensors used to monitor a workout; selectand play music for a workout; and display, store, and transmit workoutdata.

In conjunction with touch screen 112, display controller 156, opticalsensor(s) 164, optical sensor controller 158, contact/motion module 130,graphics module 132, and image management module 144, camera module 143includes executable instructions to capture still images or video(including a video stream) and store them into memory 102, modifycharacteristics of a still image or video, or delete a still image orvideo from memory 102.

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, text input module 134,and camera module 143, image management module 144 includes executableinstructions to arrange, modify (e.g., edit), or otherwise manipulate,label, delete, present (e.g., in a digital slide show or album), andstore still and/or video images.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, and textinput module 134, browser module 147 includes executable instructions tobrowse the Internet in accordance with user instructions, includingsearching, linking to, receiving, and displaying web pages or portionsthereof, as well as attachments and other files linked to web pages.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, textinput module 134, e-mail client module 140, and browser module 147,calendar module 148 includes executable instructions to create, display,modify, and store calendars and data associated with calendars (e.g.,calendar entries, to-do lists, etc.) in accordance with userinstructions.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, textinput module 134, and browser module 147, widget modules 149 aremini-applications that are, optionally, downloaded and used by a user(e.g., weather widget 149-1, stocks widget 149-2, calculator widget149-3, alarm clock widget 149-4, and dictionary widget 149-5) or createdby the user (e.g., user-created widget 149-6). In some embodiments, awidget includes an HTML (Hypertext Markup Language) file, a CSS(Cascading Style Sheets) file, and a JavaScript file. In someembodiments, a widget includes an XML (Extensible Markup Language) fileand a JavaScript file (e.g., Yahoo! Widgets).

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, textinput module 134, and browser module 147, the widget creator module 150are, optionally, used by a user to create widgets (e.g., turning auser-specified portion of a web page into a widget).

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, and text input module134, search module 151 includes executable instructions to search fortext, music, sound, image, video, and/or other files in memory 102 thatmatch one or more search criteria (e.g., one or more user-specifiedsearch terms) in accordance with user instructions.

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, audio circuitry 110,speaker 111, RF circuitry 108, and browser module 147, video and musicplayer module 152 includes executable instructions that allow the userto download and play back recorded music and other sound files stored inone or more file formats, such as MP3 or AAC files, and executableinstructions to display, present, or otherwise play back videos (e.g.,on touch screen 112 or on an external, connected display via externalport 124). In some embodiments, device 100 optionally includes thefunctionality of an MP3 player, such as an iPod (trademark of AppleInc.).

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, and text input module134, notes module 153 includes executable instructions to create andmanage notes, to-do lists, and the like in accordance with userinstructions.

In conjunction with RF circuitry 108, touch screen 112, displaycontroller 156, contact/motion module 130, graphics module 132, textinput module 134, GPS module 135, and browser module 147, map module 154are, optionally, used to receive, display, modify, and store maps anddata associated with maps (e.g., driving directions, data on stores andother points of interest at or near a particular location, and otherlocation-based data) in accordance with user instructions.

In conjunction with touch screen 112, display controller 156,contact/motion module 130, graphics module 132, audio circuitry 110,speaker 111, RF circuitry 108, text input module 134, e-mail clientmodule 140, and browser module 147, online video module 155 includesinstructions that allow the user to access, browse, receive (e.g., bystreaming and/or download), play back (e.g., on the touch screen or onan external, connected display via external port 124), send an e-mailwith a link to a particular online video, and otherwise manage onlinevideos in one or more file formats, such as H.264. In some embodiments,instant messaging module 141, rather than e-mail client module 140, isused to send a link to a particular online video. Additional descriptionof the online video application can be found in U.S. Provisional PatentApplication No. 60/936,562, “Portable Multifunction Device, Method, andGraphical User Interface for Playing Online Videos,” filed Jun. 20,2007, and U.S. patent application Ser. No. 11/968,067, “PortableMultifunction Device, Method, and Graphical User Interface for PlayingOnline Videos,” filed Dec. 31, 2007, the contents of which are herebyincorporated by reference in their entirety.

Each of the above-identified modules and applications corresponds to aset of executable instructions for performing one or more functionsdescribed above and the methods described in this application (e.g., thecomputer-implemented methods and other information processing methodsdescribed herein). These modules (e.g., sets of instructions) need notbe implemented as separate software programs, procedures, or modules,and thus various subsets of these modules are, optionally, combined orotherwise rearranged in various embodiments. For example, video playermodule is, optionally, combined with music player module into a singlemodule (e.g., video and music player module 152, FIG. 1A). In someembodiments, memory 102 optionally stores a subset of the modules anddata structures identified above. Furthermore, memory 102 optionallystores additional modules and data structures not described above.

In some embodiments, device 100 is a device where operation of apredefined set of functions on the device is performed exclusivelythrough a touch screen and/or a touchpad. By using a touch screen and/ora touchpad as the primary input control device for operation of device100, the number of physical input control devices (such as push buttons,dials, and the like) on device 100 is, optionally, reduced.

The predefined set of functions that are performed exclusively through atouch screen and/or a touchpad optionally include navigation betweenuser interfaces. In some embodiments, the touchpad, when touched by theuser, navigates device 100 to a main, home, or root menu from any userinterface that is displayed on device 100. In such embodiments, a “menubutton” is implemented using a touchpad. In some other embodiments, themenu button is a physical push button or other physical input controldevice instead of a touchpad.

FIG. 1B is a block diagram illustrating exemplary components for eventhandling in accordance with some embodiments. In some embodiments,memory 102 (FIG. 1A) or 370 (FIG. 3) includes event sorter 170 (e.g., inoperating system 126) and a respective application 136-1 (e.g., any ofthe aforementioned applications 137-151, 155, 380-390).

Event sorter 170 receives event information and determines theapplication 136-1 and application view 191 of application 136-1 to whichto deliver the event information. Event sorter 170 includes eventmonitor 171 and event dispatcher module 174. In some embodiments,application 136-1 includes application internal state 192, whichindicates the current application view(s) displayed on touch-sensitivedisplay 112 when the application is active or executing. In someembodiments, device/global internal state 157 is used by event sorter170 to determine which application(s) is (are) currently active, andapplication internal state 192 is used by event sorter 170 to determineapplication views 191 to which to deliver event information.

In some embodiments, application internal state 192 includes additionalinformation, such as one or more of: resume information to be used whenapplication 136-1 resumes execution, user interface state informationthat indicates information being displayed or that is ready for displayby application 136-1, a state queue for enabling the user to go back toa prior state or view of application 136-1, and a redo/undo queue ofprevious actions taken by the user.

Event monitor 171 receives event information from peripherals interface118. Event information includes information about a sub-event (e.g., auser touch on touch-sensitive display 112, as part of a multi-touchgesture). Peripherals interface 118 transmits information it receivesfrom I/O subsystem 106 or a sensor, such as proximity sensor 166,accelerometer(s) 168, and/or microphone 113 (through audio circuitry110). Information that peripherals interface 118 receives from I/Osubsystem 106 includes information from touch-sensitive display 112 or atouch-sensitive surface.

In some embodiments, event monitor 171 sends requests to the peripheralsinterface 118 at predetermined intervals. In response, peripheralsinterface 118 transmits event information. In other embodiments,peripherals interface 118 transmits event information only when there isa significant event (e.g., receiving an input above a predeterminednoise threshold and/or for more than a predetermined duration).

In some embodiments, event sorter 170 also includes a hit viewdetermination module 172 and/or an active event recognizer determinationmodule 173.

Hit view determination module 172 provides software procedures fordetermining where a sub-event has taken place within one or more viewswhen touch-sensitive display 112 displays more than one view. Views aremade up of controls and other elements that a user can see on thedisplay.

Another aspect of the user interface associated with an application is aset of views, sometimes herein called application views or userinterface windows, in which information is displayed and touch-basedgestures occur. The application views (of a respective application) inwhich a touch is detected optionally correspond to programmatic levelswithin a programmatic or view hierarchy of the application. For example,the lowest level view in which a touch is detected is, optionally,called the hit view, and the set of events that are recognized as properinputs are, optionally, determined based, at least in part, on the hitview of the initial touch that begins a touch-based gesture.

Hit view determination module 172 receives information related tosub-events of a touch-based gesture. When an application has multipleviews organized in a hierarchy, hit view determination module 172identifies a hit view as the lowest view in the hierarchy which shouldhandle the sub-event. In most circumstances, the hit view is the lowestlevel view in which an initiating sub-event occurs (e.g., the firstsub-event in the sequence of sub-events that form an event or potentialevent). Once the hit view is identified by the hit view determinationmodule 172, the hit view typically receives all sub-events related tothe same touch or input source for which it was identified as the hitview.

Active event recognizer determination module 173 determines which viewor views within a view hierarchy should receive a particular sequence ofsub-events. In some embodiments, active event recognizer determinationmodule 173 determines that only the hit view should receive a particularsequence of sub-events. In other embodiments, active event recognizerdetermination module 173 determines that all views that include thephysical location of a sub-event are actively involved views, andtherefore determines that all actively involved views should receive aparticular sequence of sub-events. In other embodiments, even if touchsub-events were entirely confined to the area associated with oneparticular view, views higher in the hierarchy would still remain asactively involved views.

Event dispatcher module 174 dispatches the event information to an eventrecognizer (e.g., event recognizer 180). In embodiments including activeevent recognizer determination module 173, event dispatcher module 174delivers the event information to an event recognizer determined byactive event recognizer determination module 173. In some embodiments,event dispatcher module 174 stores in an event queue the eventinformation, which is retrieved by a respective event receiver 182.

In some embodiments, operating system 126 includes event sorter 170.Alternatively, application 136-1 includes event sorter 170. In yet otherembodiments, event sorter 170 is a stand-alone module, or a part ofanother module stored in memory 102, such as contact/motion module 130.

In some embodiments, application 136-1 includes a plurality of eventhandlers 190 and one or more application views 191, each of whichincludes instructions for handling touch events that occur within arespective view of the application's user interface. Each applicationview 191 of the application 136-1 includes one or more event recognizers180. Typically, a respective application view 191 includes a pluralityof event recognizers 180. In other embodiments, one or more of eventrecognizers 180 are part of a separate module, such as a user interfacekit or a higher level object from which application 136-1 inheritsmethods and other properties. In some embodiments, a respective eventhandler 190 includes one or more of: data updater 176, object updater177, GUI updater 178, and/or event data 179 received from event sorter170. Event handler 190 optionally utilizes or calls data updater 176,object updater 177, or GUI updater 178 to update the applicationinternal state 192. Alternatively, one or more of the application views191 include one or more respective event handlers 190. Also, in someembodiments, one or more of data updater 176, object updater 177, andGUI updater 178 are included in a respective application view 191.

A respective event recognizer 180 receives event information (e.g.,event data 179) from event sorter 170 and identifies an event from theevent information. Event recognizer 180 includes event receiver 182 andevent comparator 184. In some embodiments, event recognizer 180 alsoincludes at least a subset of: metadata 183, and event deliveryinstructions 188 (which optionally include sub-event deliveryinstructions).

Event receiver 182 receives event information from event sorter 170. Theevent information includes information about a sub-event, for example, atouch or a touch movement. Depending on the sub-event, the eventinformation also includes additional information, such as location ofthe sub-event. When the sub-event concerns motion of a touch, the eventinformation optionally also includes speed and direction of thesub-event. In some embodiments, events include rotation of the devicefrom one orientation to another (e.g., from a portrait orientation to alandscape orientation, or vice versa), and the event informationincludes corresponding information about the current orientation (alsocalled device attitude) of the device.

Event comparator 184 compares the event information to predefined eventor sub-event definitions and, based on the comparison, determines anevent or sub-event, or determines or updates the state of an event orsub-event. In some embodiments, event comparator 184 includes eventdefinitions 186. Event definitions 186 contain definitions of events(e.g., predefined sequences of sub-events), for example, event 1(187-1), event 2 (187-2), and others. In some embodiments, sub-events inan event (187) include, for example, touch begin, touch end, touchmovement, touch cancellation, and multiple touching. In one example, thedefinition for event 1 (187-1) is a double tap on a displayed object.The double tap, for example, comprises a first touch (touch begin) onthe displayed object for a predetermined phase, a first liftoff (touchend) for a predetermined phase, a second touch (touch begin) on thedisplayed object for a predetermined phase, and a second liftoff (touchend) for a predetermined phase. In another example, the definition forevent 2 (187-2) is a dragging on a displayed object. The dragging, forexample, comprises a touch (or contact) on the displayed object for apredetermined phase, a movement of the touch across touch-sensitivedisplay 112, and liftoff of the touch (touch end). In some embodiments,the event also includes information for one or more associated eventhandlers 190.

In some embodiments, event definition 187 includes a definition of anevent for a respective user-interface object. In some embodiments, eventcomparator 184 performs a hit test to determine which user-interfaceobject is associated with a sub-event. For example, in an applicationview in which three user-interface objects are displayed ontouch-sensitive display 112, when a touch is detected on touch-sensitivedisplay 112, event comparator 184 performs a hit test to determine whichof the three user-interface objects is associated with the touch(sub-event). If each displayed object is associated with a respectiveevent handler 190, the event comparator uses the result of the hit testto determine which event handler 190 should be activated. For example,event comparator 184 selects an event handler associated with thesub-event and the object triggering the hit test.

In some embodiments, the definition for a respective event (187) alsoincludes delayed actions that delay delivery of the event informationuntil after it has been determined whether the sequence of sub-eventsdoes or does not correspond to the event recognizer's event type.

When a respective event recognizer 180 determines that the series ofsub-events do not match any of the events in event definitions 186, therespective event recognizer 180 enters an event impossible, eventfailed, or event ended state, after which it disregards subsequentsub-events of the touch-based gesture. In this situation, other eventrecognizers, if any, that remain active for the hit view continue totrack and process sub-events of an ongoing touch-based gesture.

In some embodiments, a respective event recognizer 180 includes metadata183 with configurable properties, flags, and/or lists that indicate howthe event delivery system should perform sub-event delivery to activelyinvolved event recognizers. In some embodiments, metadata 183 includesconfigurable properties, flags, and/or lists that indicate how eventrecognizers interact, or are enabled to interact, with one another. Insome embodiments, metadata 183 includes configurable properties, flags,and/or lists that indicate whether sub-events are delivered to varyinglevels in the view or programmatic hierarchy.

In some embodiments, a respective event recognizer 180 activates eventhandler 190 associated with an event when one or more particularsub-events of an event are recognized. In some embodiments, a respectiveevent recognizer 180 delivers event information associated with theevent to event handler 190. Activating an event handler 190 is distinctfrom sending (and deferred sending) sub-events to a respective hit view.In some embodiments, event recognizer 180 throws a flag associated withthe recognized event, and event handler 190 associated with the flagcatches the flag and performs a predefined process.

In some embodiments, event delivery instructions 188 include sub-eventdelivery instructions that deliver event information about a sub-eventwithout activating an event handler. Instead, the sub-event deliveryinstructions deliver event information to event handlers associated withthe series of sub-events or to actively involved views. Event handlersassociated with the series of sub-events or with actively involved viewsreceive the event information and perform a predetermined process.

In some embodiments, data updater 176 creates and updates data used inapplication 136-1. For example, data updater 176 updates the telephonenumber used in contacts module 137, or stores a video file used in videoplayer module. In some embodiments, object updater 177 creates andupdates objects used in application 136-1. For example, object updater177 creates a new user-interface object or updates the position of auser-interface object. GUI updater 178 updates the GUI. For example, GUIupdater 178 prepares display information and sends it to graphics module132 for display on a touch-sensitive display.

In some embodiments, event handler(s) 190 includes or has access to dataupdater 176, object updater 177, and GUI updater 178. In someembodiments, data updater 176, object updater 177, and GUI updater 178are included in a single module of a respective application 136-1 orapplication view 191. In other embodiments, they are included in two ormore software modules.

It shall be understood that the foregoing discussion regarding eventhandling of user touches on touch-sensitive displays also applies toother forms of user inputs to operate multifunction devices 100 withinput devices, not all of which are initiated on touch screens. Forexample, mouse movement and mouse button presses, optionally coordinatedwith single or multiple keyboard presses or holds; contact movementssuch as taps, drags, scrolls, etc. on touchpads; pen stylus inputs;movement of the device; oral instructions; detected eye movements;biometric inputs; and/or any combination thereof are optionally utilizedas inputs corresponding to sub-events which define an event to berecognized.

FIG. 2 illustrates a portable multifunction device 100 having a touchscreen 112 in accordance with some embodiments. The touch screenoptionally displays one or more graphics within user interface (UI) 200.In this embodiment, as well as others described below, a user is enabledto select one or more of the graphics by making a gesture on thegraphics, for example, with one or more fingers 202 (not drawn to scalein the figure) or one or more styluses 203 (not drawn to scale in thefigure). In some embodiments, selection of one or more graphics occurswhen the user breaks contact with the one or more graphics. In someembodiments, the gesture optionally includes one or more taps, one ormore swipes (from left to right, right to left, upward and/or downward),and/or a rolling of a finger (from right to left, left to right, upwardand/or downward) that has made contact with device 100. In someimplementations or circumstances, inadvertent contact with a graphicdoes not select the graphic. For example, a swipe gesture that sweepsover an application icon optionally does not select the correspondingapplication when the gesture corresponding to selection is a tap.

Device 100 optionally also include one or more physical buttons, such as“home” or menu button 204. As described previously, menu button 204 is,optionally, used to navigate to any application 136 in a set ofapplications that are, optionally, executed on device 100.Alternatively, in some embodiments, the menu button is implemented as asoft key in a GUI displayed on touch screen 112.

In some embodiments, device 100 includes touch screen 112, menu button204, push button 206 for powering the device on/off and locking thedevice, volume adjustment button(s) 208, subscriber identity module(SIM) card slot 210, headset jack 212, and docking/charging externalport 124. Push button 206 is, optionally, used to turn the power on/offon the device by depressing the button and holding the button in thedepressed state for a predefined time interval; to lock the device bydepressing the button and releasing the button before the predefinedtime interval has elapsed; and/or to unlock the device or initiate anunlock process. In an alternative embodiment, device 100 also acceptsverbal input for activation or deactivation of some functions throughmicrophone 113. Device 100 also, optionally, includes one or morecontact intensity sensors 165 for detecting intensity of contacts ontouch screen 112 and/or one or more tactile output generators 167 forgenerating tactile outputs for a user of device 100.

FIG. 3 is a block diagram of an exemplary multifunction device with adisplay and a touch-sensitive surface in accordance with someembodiments. Device 300 need not be portable. In some embodiments,device 300 is a laptop computer, a desktop computer, a tablet computer,a multimedia player device, a navigation device, an educational device(such as a child's learning toy), a gaming system, or a control device(e.g., a home or industrial controller). Device 300 typically includesone or more processing units (CPUs) 310, one or more network or othercommunications interfaces 360, memory 370, and one or more communicationbuses 320 for interconnecting these components. Communication buses 320optionally include circuitry (sometimes called a chipset) thatinterconnects and controls communications between system components.Device 300 includes input/output (I/O) interface 330 comprising display340, which is typically a touch screen display. I/O interface 330 alsooptionally includes a keyboard and/or mouse (or other pointing device)350 and touchpad 355, tactile output generator 357 for generatingtactile outputs on device 300 (e.g., similar to tactile outputgenerator(s) 167 described above with reference to FIG. 1A), sensors 359(e.g., optical, acceleration, proximity, touch-sensitive, and/or contactintensity sensors similar to contact intensity sensor(s) 165 describedabove with reference to FIG. 1A). Memory 370 includes high-speed randomaccess memory, such as DRAM, SRAM, DDR RAM, or other random access solidstate memory devices; and optionally includes non-volatile memory, suchas one or more magnetic disk storage devices, optical disk storagedevices, flash memory devices, or other non-volatile solid state storagedevices. Memory 370 optionally includes one or more storage devicesremotely located from CPU(s) 310. In some embodiments, memory 370 storesprograms, modules, and data structures analogous to the programs,modules, and data structures stored in memory 102 of portablemultifunction device 100 (FIG. 1A), or a subset thereof. Furthermore,memory 370 optionally stores additional programs, modules, and datastructures not present in memory 102 of portable multifunction device100. For example, memory 370 of device 300 optionally stores drawingmodule 380, presentation module 382, word processing module 384, websitecreation module 386, disk authoring module 388, and/or spreadsheetmodule 390, while memory 102 of portable multifunction device 100 (FIG.1A) optionally does not store these modules.

Each of the above-identified elements in FIG. 3 is, optionally, storedin one or more of the previously mentioned memory devices. Each of theabove-identified modules corresponds to a set of instructions forperforming a function described above. The above-identified modules orprograms (e.g., sets of instructions) need not be implemented asseparate software programs, procedures, or modules, and thus varioussubsets of these modules are, optionally, combined or otherwiserearranged in various embodiments. In some embodiments, memory 370optionally stores a subset of the modules and data structures identifiedabove. Furthermore, memory 370 optionally stores additional modules anddata structures not described above.

Attention is now directed towards embodiments of user interfaces thatare, optionally, implemented on, for example, portable multifunctiondevice 100.

FIG. 4A illustrates an exemplary user interface for a menu ofapplications on portable multifunction device 100 in accordance withsome embodiments. Similar user interfaces are, optionally, implementedon device 300. In some embodiments, user interface 400 includes thefollowing elements, or a subset or superset thereof:

-   -   Signal strength indicator(s) 402 for wireless communication(s),        such as cellular and Wi-Fi signals;    -   Time 404;    -   Bluetooth indicator 405;    -   Battery status indicator 406;    -   Tray 408 with icons for frequently used applications, such as:        -   Icon 416 for telephone module 138, labeled “Phone,” which            optionally includes an indicator 414 of the number of missed            calls or voicemail messages;        -   Icon 418 for e-mail client module 140, labeled “Mail,” which            optionally includes an indicator 410 of the number of unread            e-mails;        -   Icon 420 for browser module 147, labeled “Browser;” and        -   Icon 422 for video and music player module 152, also            referred to as iPod (trademark of Apple Inc.) module 152,            labeled “iPod;” and    -   Icons for other applications, such as:        -   Icon 424 for IM module 141, labeled “Messages;”        -   Icon 426 for calendar module 148, labeled “Calendar;”        -   Icon 428 for image management module 144, labeled “Photos;”        -   Icon 430 for camera module 143, labeled “Camera;”        -   Icon 432 for online video module 155, labeled “Online            Video;”        -   Icon 434 for stocks widget 149-2, labeled “Stocks;”        -   Icon 436 for map module 154, labeled “Maps;”        -   Icon 438 for weather widget 149-1, labeled “Weather;”        -   Icon 440 for alarm clock widget 149-4, labeled “Clock;”        -   Icon 442 for workout support module 142, labeled “Workout            Support;”        -   Icon 444 for notes module 153, labeled “Notes;” and        -   Icon 446 for a settings application or module, labeled            “Settings,” which provides access to settings for device 100            and its various applications 136.

It should be noted that the icon labels illustrated in FIG. 4A aremerely exemplary. For example, icon 422 for video and music playermodule 152 is labeled “Music” or “Music Player.” Other labels are,optionally, used for various application icons. In some embodiments, alabel for a respective application icon includes a name of anapplication corresponding to the respective application icon. In someembodiments, a label for a particular application icon is distinct froma name of an application corresponding to the particular applicationicon.

FIG. 4B illustrates an exemplary user interface on a device (e.g.,device 300, FIG. 3) with a touch-sensitive surface 451 (e.g., a tabletor touchpad 355, FIG. 3) that is separate from the display 450 (e.g.,touch screen display 112). Device 300 also, optionally, includes one ormore contact intensity sensors (e.g., one or more of sensors 359) fordetecting intensity of contacts on touch-sensitive surface 451 and/orone or more tactile output generators 357 for generating tactile outputsfor a user of device 300.

Although some of the examples that follow will be given with referenceto inputs on touch screen display 112 (where the touch-sensitive surfaceand the display are combined), in some embodiments, the device detectsinputs on a touch-sensitive surface that is separate from the display,as shown in FIG. 4B. In some embodiments, the touch-sensitive surface(e.g., 451 in FIG. 4B) has a primary axis (e.g., 452 in FIG. 4B) thatcorresponds to a primary axis (e.g., 453 in FIG. 4B) on the display(e.g., 450). In accordance with these embodiments, the device detectscontacts (e.g., 460 and 462 in FIG. 4B) with the touch-sensitive surface451 at locations that correspond to respective locations on the display(e.g., in FIG. 4B, 460 corresponds to 468 and 462 corresponds to 470).In this way, user inputs (e.g., contacts 460 and 462, and movementsthereof) detected by the device on the touch-sensitive surface (e.g.,451 in FIG. 4B) are used by the device to manipulate the user interfaceon the display (e.g., 450 in FIG. 4B) of the multifunction device whenthe touch-sensitive surface is separate from the display. It should beunderstood that similar methods are, optionally, used for other userinterfaces described herein.

Additionally, while the following examples are given primarily withreference to finger inputs (e.g., finger contacts, finger tap gestures,finger swipe gestures), it should be understood that, in someembodiments, one or more of the finger inputs are replaced with inputfrom another input device (e.g., a mouse-based input or stylus input).For example, a swipe gesture is, optionally, replaced with a mouse click(e.g., instead of a contact) followed by movement of the cursor alongthe path of the swipe (e.g., instead of movement of the contact). Asanother example, a tap gesture is, optionally, replaced with a mouseclick while the cursor is located over the location of the tap gesture(e.g., instead of detection of the contact followed by ceasing to detectthe contact). Similarly, when multiple user inputs are simultaneouslydetected, it should be understood that multiple computer mice are,optionally, used simultaneously, or a mouse and finger contacts are,optionally, used simultaneously.

FIG. 5A illustrates exemplary personal electronic device 500. Device 500includes body 502. In some embodiments, device 500 can include some orall of the features described with respect to devices 100 and 300 (e.g.,FIGS. 1A-4B). In some embodiments, device 500 has touch-sensitivedisplay screen 504, hereafter touch screen 504. Alternatively, or inaddition to touch screen 504, device 500 has a display and atouch-sensitive surface. As with devices 100 and 300, in someembodiments, touch screen 504 (or the touch-sensitive surface)optionally includes one or more intensity sensors for detectingintensity of contacts (e.g., touches) being applied. The one or moreintensity sensors of touch screen 504 (or the touch-sensitive surface)can provide output data that represents the intensity of touches. Theuser interface of device 500 can respond to touches based on theirintensity, meaning that touches of different intensities can invokedifferent user interface operations on device 500.

Exemplary techniques for detecting and processing touch intensity arefound, for example, in related applications: International PatentApplication Serial No. PCT/US2013/040061, titled “Device, Method, andGraphical User Interface for Displaying User Interface ObjectsCorresponding to an Application,” filed May 8, 2013, published as WIPOPublication No. WO/2013/169849, and International Patent ApplicationSerial No. PCT/US2013/069483, titled “Device, Method, and Graphical UserInterface for Transitioning Between Touch Input to Display OutputRelationships,” filed Nov. 11, 2013, published as WIPO Publication No.WO/2014/105276, each of which is hereby incorporated by reference intheir entirety.

In some embodiments, device 500 has one or more input mechanisms 506 and508. Input mechanisms 506 and 508, if included, can be physical.Examples of physical input mechanisms include push buttons and rotatablemechanisms. In some embodiments, device 500 has one or more attachmentmechanisms. Such attachment mechanisms, if included, can permitattachment of device 500 with, for example, hats, eyewear, earrings,necklaces, shirts, jackets, bracelets, watch straps, chains, trousers,belts, shoes, purses, backpacks, and so forth. These attachmentmechanisms permit device 500 to be worn by a user.

FIG. 5B depicts exemplary personal electronic device 500. In someembodiments, device 500 can include some or all of the componentsdescribed with respect to FIGS. 1A, 1B, and 3. Device 500 has bus 512that operatively couples I/O section 514 with one or more computerprocessors 516 and memory 518. I/O section 514 can be connected todisplay 504, which can have touch-sensitive component 522 and,optionally, intensity sensor 524 (e.g., contact intensity sensor). Inaddition, I/O section 514 can be connected with communication unit 530for receiving application and operating system data, using Wi-Fi,Bluetooth, near field communication (NFC), cellular, and/or otherwireless communication techniques. Device 500 can include inputmechanisms 506 and/or 508. Input mechanism 506 is, optionally, arotatable input device or a depressible and rotatable input device, forexample. Input mechanism 508 is, optionally, a button, in some examples.

Input mechanism 508 is, optionally, a microphone, in some examples.Personal electronic device 500 optionally includes various sensors, suchas GPS sensor 532, accelerometer 534, directional sensor 540 (e.g.,compass), gyroscope 536, motion sensor 538, and/or a combinationthereof, all of which can be operatively connected to I/O section 514.

Memory 518 of personal electronic device 500 can include one or morenon-transitory computer-readable storage mediums, for storingcomputer-executable instructions, which, when executed by one or morecomputer processors 516, for example, can cause the computer processorsto perform the techniques described below, including processes 800(FIGS. 8A-8B), 1000 (FIG. 10), 1200 (FIG. 12), 1400 (FIG. 14), and 1600(FIG. 16). A computer-readable storage medium can be any medium that cantangibly contain or store computer-executable instructions for use by orin connection with the instruction execution system, apparatus, ordevice. In some examples, the storage medium is a transitorycomputer-readable storage medium. In some examples, the storage mediumis a non-transitory computer-readable storage medium. The non-transitorycomputer-readable storage medium can include, but is not limited to,magnetic, optical, and/or semiconductor storages. Examples of suchstorage include magnetic disks, optical discs based on CD, DVD, orBlu-ray technologies, as well as persistent solid-state memory such asflash, solid-state drives, and the like. Personal electronic device 500is not limited to the components and configuration of FIG. 5B, but caninclude other or additional components in multiple configurations.

As used here, the term “affordance” refers to a user-interactivegraphical user interface object that is, optionally, displayed on thedisplay screen of devices 100, 300, and/or 500 (FIGS. 1A, 3, and 5A-5B).For example, an image (e.g., icon), a button, and text (e.g., hyperlink)each optionally constitute an affordance.

As used herein, the term “focus selector” refers to an input elementthat indicates a current part of a user interface with which a user isinteracting. In some implementations that include a cursor or otherlocation marker, the cursor acts as a “focus selector” so that when aninput (e.g., a press input) is detected on a touch-sensitive surface(e.g., touchpad 355 in FIG. 3 or touch-sensitive surface 451 in FIG. 4B)while the cursor is over a particular user interface element (e.g., abutton, window, slider, or other user interface element), the particularuser interface element is adjusted in accordance with the detectedinput. In some implementations that include a touch screen display(e.g., touch-sensitive display system 112 in FIG. 1A or touch screen 112in FIG. 4A) that enables direct interaction with user interface elementson the touch screen display, a detected contact on the touch screen actsas a “focus selector” so that when an input (e.g., a press input by thecontact) is detected on the touch screen display at a location of aparticular user interface element (e.g., a button, window, slider, orother user interface element), the particular user interface element isadjusted in accordance with the detected input. In some implementations,focus is moved from one region of a user interface to another region ofthe user interface without corresponding movement of a cursor ormovement of a contact on a touch screen display (e.g., by using a tabkey or arrow keys to move focus from one button to another button); inthese implementations, the focus selector moves in accordance withmovement of focus between different regions of the user interface.Without regard to the specific form taken by the focus selector, thefocus selector is generally the user interface element (or contact on atouch screen display) that is controlled by the user so as tocommunicate the user's intended interaction with the user interface(e.g., by indicating, to the device, the element of the user interfacewith which the user is intending to interact). For example, the locationof a focus selector (e.g., a cursor, a contact, or a selection box) overa respective button while a press input is detected on thetouch-sensitive surface (e.g., a touchpad or touch screen) will indicatethat the user is intending to activate the respective button (as opposedto other user interface elements shown on a display of the device).

As used in the specification and claims, the term “characteristicintensity” of a contact refers to a characteristic of the contact basedon one or more intensities of the contact. In some embodiments, thecharacteristic intensity is based on multiple intensity samples. Thecharacteristic intensity is, optionally, based on a predefined number ofintensity samples, or a set of intensity samples collected during apredetermined time period (e.g., 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10seconds) relative to a predefined event (e.g., after detecting thecontact, prior to detecting liftoff of the contact, before or afterdetecting a start of movement of the contact, prior to detecting an endof the contact, before or after detecting an increase in intensity ofthe contact, and/or before or after detecting a decrease in intensity ofthe contact). A characteristic intensity of a contact is, optionally,based on one or more of: a maximum value of the intensities of thecontact, a mean value of the intensities of the contact, an averagevalue of the intensities of the contact, a top 10 percentile value ofthe intensities of the contact, a value at the half maximum of theintensities of the contact, a value at the 90 percent maximum of theintensities of the contact, or the like. In some embodiments, theduration of the contact is used in determining the characteristicintensity (e.g., when the characteristic intensity is an average of theintensity of the contact over time). In some embodiments, thecharacteristic intensity is compared to a set of one or more intensitythresholds to determine whether an operation has been performed by auser. For example, the set of one or more intensity thresholdsoptionally includes a first intensity threshold and a second intensitythreshold. In this example, a contact with a characteristic intensitythat does not exceed the first threshold results in a first operation, acontact with a characteristic intensity that exceeds the first intensitythreshold and does not exceed the second intensity threshold results ina second operation, and a contact with a characteristic intensity thatexceeds the second threshold results in a third operation. In someembodiments, a comparison between the characteristic intensity and oneor more thresholds is used to determine whether or not to perform one ormore operations (e.g., whether to perform a respective operation orforgo performing the respective operation), rather than being used todetermine whether to perform a first operation or a second operation.

FIG. 5C illustrates detecting a plurality of contacts 552A-552E ontouch-sensitive display screen 504 with a plurality of intensity sensors524A-524D. FIG. 5C additionally includes intensity diagrams that showthe current intensity measurements of the intensity sensors 524A-524Drelative to units of intensity. In this example, the intensitymeasurements of intensity sensors 524A and 524D are each 9 units ofintensity, and the intensity measurements of intensity sensors 524B and524C are each 7 units of intensity. In some implementations, anaggregate intensity is the sum of the intensity measurements of theplurality of intensity sensors 524A-524D, which in this example is 32intensity units. In some embodiments, each contact is assigned arespective intensity that is a portion of the aggregate intensity. FIG.5D illustrates assigning the aggregate intensity to contacts 552A-552Ebased on their distance from the center of force 554. In this example,each of contacts 552A, 552B, and 552E are assigned an intensity ofcontact of 8 intensity units of the aggregate intensity, and each ofcontacts 552C and 552D are assigned an intensity of contact of 4intensity units of the aggregate intensity. More generally, in someimplementations, each contact j is assigned a respective intensity Ijthat is a portion of the aggregate intensity, A, in accordance with apredefined mathematical function, Ij=A·(Dj/ΣDi), where Dj is thedistance of the respective contact j to the center of force, and ΣDi isthe sum of the distances of all the respective contacts (e.g., i=1 tolast) to the center of force. The operations described with reference toFIGS. 5C-5D can be performed using an electronic device similar oridentical to device 100, 300, or 500. In some embodiments, acharacteristic intensity of a contact is based on one or moreintensities of the contact. In some embodiments, the intensity sensorsare used to determine a single characteristic intensity (e.g., a singlecharacteristic intensity of a single contact). It should be noted thatthe intensity diagrams are not part of a displayed user interface, butare included in FIGS. 5C-5D to aid the reader.

In some embodiments, a portion of a gesture is identified for purposesof determining a characteristic intensity. For example, atouch-sensitive surface optionally receives a continuous swipe contacttransitioning from a start location and reaching an end location, atwhich point the intensity of the contact increases. In this example, thecharacteristic intensity of the contact at the end location is,optionally, based on only a portion of the continuous swipe contact, andnot the entire swipe contact (e.g., only the portion of the swipecontact at the end location). In some embodiments, a smoothing algorithmis, optionally, applied to the intensities of the swipe contact prior todetermining the characteristic intensity of the contact. For example,the smoothing algorithm optionally includes one or more of: anunweighted sliding-average smoothing algorithm, a triangular smoothingalgorithm, a median filter smoothing algorithm, and/or an exponentialsmoothing algorithm. In some circumstances, these smoothing algorithmseliminate narrow spikes or dips in the intensities of the swipe contactfor purposes of determining a characteristic intensity.

The intensity of a contact on the touch-sensitive surface is,optionally, characterized relative to one or more intensity thresholds,such as a contact-detection intensity threshold, a light press intensitythreshold, a deep press intensity threshold, and/or one or more otherintensity thresholds. In some embodiments, the light press intensitythreshold corresponds to an intensity at which the device will performoperations typically associated with clicking a button of a physicalmouse or a trackpad. In some embodiments, the deep press intensitythreshold corresponds to an intensity at which the device will performoperations that are different from operations typically associated withclicking a button of a physical mouse or a trackpad. In someembodiments, when a contact is detected with a characteristic intensitybelow the light press intensity threshold (e.g., and above a nominalcontact-detection intensity threshold below which the contact is nolonger detected), the device will move a focus selector in accordancewith movement of the contact on the touch-sensitive surface withoutperforming an operation associated with the light press intensitythreshold or the deep press intensity threshold. Generally, unlessotherwise stated, these intensity thresholds are consistent betweendifferent sets of user interface figures.

An increase of characteristic intensity of the contact from an intensitybelow the light press intensity threshold to an intensity between thelight press intensity threshold and the deep press intensity thresholdis sometimes referred to as a “light press” input. An increase ofcharacteristic intensity of the contact from an intensity below the deeppress intensity threshold to an intensity above the deep press intensitythreshold is sometimes referred to as a “deep press” input. An increaseof characteristic intensity of the contact from an intensity below thecontact-detection intensity threshold to an intensity between thecontact-detection intensity threshold and the light press intensitythreshold is sometimes referred to as detecting the contact on thetouch-surface. A decrease of characteristic intensity of the contactfrom an intensity above the contact-detection intensity threshold to anintensity below the contact-detection intensity threshold is sometimesreferred to as detecting liftoff of the contact from the touch-surface.In some embodiments, the contact-detection intensity threshold is zero.In some embodiments, the contact-detection intensity threshold isgreater than zero.

In some embodiments described herein, one or more operations areperformed in response to detecting a gesture that includes a respectivepress input or in response to detecting the respective press inputperformed with a respective contact (or a plurality of contacts), wherethe respective press input is detected based at least in part ondetecting an increase in intensity of the contact (or plurality ofcontacts) above a press-input intensity threshold. In some embodiments,the respective operation is performed in response to detecting theincrease in intensity of the respective contact above the press-inputintensity threshold (e.g., a “down stroke” of the respective pressinput). In some embodiments, the press input includes an increase inintensity of the respective contact above the press-input intensitythreshold and a subsequent decrease in intensity of the contact belowthe press-input intensity threshold, and the respective operation isperformed in response to detecting the subsequent decrease in intensityof the respective contact below the press-input threshold (e.g., an “upstroke” of the respective press input).

FIGS. 5E-5H illustrate detection of a gesture that includes a pressinput that corresponds to an increase in intensity of a contact 562 froman intensity below a light press intensity threshold (e.g., “IT_(L)”) inFIG. 5E, to an intensity above a deep press intensity threshold (e.g.,“IT_(D)”) in FIG. 5H. The gesture performed with contact 562 is detectedon touch-sensitive surface 560 while cursor 576 is displayed overapplication icon 572B corresponding to App 2, on a displayed userinterface 570 that includes application icons 572A-572D displayed inpredefined region 574. In some embodiments, the gesture is detected ontouch-sensitive display 504. The intensity sensors detect the intensityof contacts on touch-sensitive surface 560. The device determines thatthe intensity of contact 562 peaked above the deep press intensitythreshold (e.g., “IT_(D)”). Contact 562 is maintained on touch-sensitivesurface 560. In response to the detection of the gesture, and inaccordance with contact 562 having an intensity that goes above the deeppress intensity threshold (e.g., “IT_(D)”) during the gesture,reduced-scale representations 578A-578C (e.g., thumbnails) of recentlyopened documents for App 2 are displayed, as shown in FIGS. 5F-5H. Insome embodiments, the intensity, which is compared to the one or moreintensity thresholds, is the characteristic intensity of a contact. Itshould be noted that the intensity diagram for contact 562 is not partof a displayed user interface, but is included in FIGS. 5E-5H to aid thereader.

In some embodiments, the display of representations 578A-578C includesan animation. For example, representation 578A is initially displayed inproximity of application icon 572B, as shown in FIG. 5F. As theanimation proceeds, representation 578A moves upward and representation578B is displayed in proximity of application icon 572B, as shown inFIG. 5G. Then, representations 578A moves upward, 578B moves upwardtoward representation 578A, and representation 578C is displayed inproximity of application icon 572B, as shown in FIG. 5H. Representations578A-578C form an array above icon 572B. In some embodiments, theanimation progresses in accordance with an intensity of contact 562, asshown in FIGS. 5F-5G, where the representations 578A-578C appear andmove upwards as the intensity of contact 562 increases toward the deeppress intensity threshold (e.g., “IT_(D)”). In some embodiments, theintensity, on which the progress of the animation is based, is thecharacteristic intensity of the contact. The operations described withreference to FIGS. 5E-5H can be performed using an electronic devicesimilar or identical to device 100, 300, or 500.

In some embodiments, the device employs intensity hysteresis to avoidaccidental inputs sometimes termed “jitter,” where the device defines orselects a hysteresis intensity threshold with a predefined relationshipto the press-input intensity threshold (e.g., the hysteresis intensitythreshold is X intensity units lower than the press-input intensitythreshold or the hysteresis intensity threshold is 75%, 90%, or somereasonable proportion of the press-input intensity threshold). Thus, insome embodiments, the press input includes an increase in intensity ofthe respective contact above the press-input intensity threshold and asubsequent decrease in intensity of the contact below the hysteresisintensity threshold that corresponds to the press-input intensitythreshold, and the respective operation is performed in response todetecting the subsequent decrease in intensity of the respective contactbelow the hysteresis intensity threshold (e.g., an “up stroke” of therespective press input). Similarly, in some embodiments, the press inputis detected only when the device detects an increase in intensity of thecontact from an intensity at or below the hysteresis intensity thresholdto an intensity at or above the press-input intensity threshold and,optionally, a subsequent decrease in intensity of the contact to anintensity at or below the hysteresis intensity, and the respectiveoperation is performed in response to detecting the press input (e.g.,the increase in intensity of the contact or the decrease in intensity ofthe contact, depending on the circumstances).

For ease of explanation, the descriptions of operations performed inresponse to a press input associated with a press-input intensitythreshold or in response to a gesture including the press input are,optionally, triggered in response to detecting either: an increase inintensity of a contact above the press-input intensity threshold, anincrease in intensity of a contact from an intensity below thehysteresis intensity threshold to an intensity above the press-inputintensity threshold, a decrease in intensity of the contact below thepress-input intensity threshold, and/or a decrease in intensity of thecontact below the hysteresis intensity threshold corresponding to thepress-input intensity threshold. Additionally, in examples where anoperation is described as being performed in response to detecting adecrease in intensity of a contact below the press-input intensitythreshold, the operation is, optionally, performed in response todetecting a decrease in intensity of the contact below a hysteresisintensity threshold corresponding to, and lower than, the press-inputintensity threshold.

FIG. 5I illustrates exemplary electronic device 580. Device 580 includesbody 580A. In some embodiments, device 580 can include some or all ofthe features described with respect to devices 100, 300, and 500 (e.g.,FIGS. 1A-5B). In some embodiments, device 580 has one or more speakers580B (concealed in body 580A), one or more microphones 580C, one or moretouch-sensitive surfaces 580D, and one or more displays 580E.Alternatively, or in addition to a display and touch-sensitive surface580D, the device has a touch-sensitive display (also referred to as atouchscreen). As with devices 100, 300, and 500, in some embodiments,touch-sensitive surface 580D (or the touch screen) optionally includesone or more intensity sensors for detecting intensity of contacts (e.g.,touches) being applied. The one or more intensity sensors oftouch-sensitive surface 580D (or the touchscreen) can provide outputdata that represents the intensity of touches. The user interface ofdevice 580 can respond to touches based on their intensity, meaning thattouches of different intensities can invoke different user interfaceoperations on device 580. In some embodiments, the one or more displays580E are one or more light-emitting diodes (LEDs). For example, adisplay can be a single LED, an LED cluster (e.g., a red, a green, and ablue LED), a plurality of discrete LEDs, a plurality of discrete LEDclusters, or other arrangement of one or more LEDs. For example, thedisplay 580E can be an array of nine discrete LED clusters arranged in acircular shape (e.g., a ring). In some examples, the one or moredisplays are comprised of one or more of another type of light-emittingelements.

FIG. 5J depicts exemplary personal electronic device 580. In someembodiments, device 580 can include some or all of the componentsdescribed with respect to FIGS. 1A, 1B, 3, and 5A-5B. Device 580 has bus592 that operatively couples I/O section 594 with one or more computerprocessors 596 and memory 598. I/O section 594 can be connected todisplay 582, which can have touch-sensitive component 584 and,optionally, intensity sensor 585 (e.g., contact intensity sensor). Insome embodiments, touch-sensitive component 584 is a separate componentthan display 582. In addition, I/O section 594 can be connected withcommunication unit 590 for receiving application and operating systemdata, using Wi-Fi, Bluetooth, near field communication (NFC), cellular,and/or other wireless communication techniques. Device 580 can includeinput mechanisms 588. Input mechanism 588 is, optionally, a button, insome examples. Input mechanism 588 is, optionally, a microphone, in someexamples. Input mechanism 588 is, optionally, a plurality of microphones(e.g., a microphone array).

Electronic device 580 includes speaker 586 for outputting audio. Device580 can include audio circuitry (e.g., in I/O section 594) that receivesaudio data, converts the audio data to an electrical signal, andtransmits the electrical signal to speaker 586. Speaker 586 converts theelectrical signal to human-audible sound waves. The audio circuitry(e.g., in I/O section 594) also receives electrical signals converted bya microphone (e.g., input mechanism 588) from sound waves. The audiocircuitry (e.g., in I/O section 594) converts the electrical signal toaudio data. Audio data is, optionally, retrieved from and/or transmittedto memory 598 and/or RF circuitry (e.g., in communication unit 590) byI/O section 594.

Memory 598 of personal electronic device 580 can include one or morenon-transitory computer-readable storage mediums, for storingcomputer-executable instructions, which, when executed by one or morecomputer processors 596, for example, can cause the computer processorsto perform the techniques described below, including processes 800(FIGS. 8A-8B), 1000 (FIG. 10), 1200 (FIG. 12), 1400 (FIG. 14), and 1600(FIG. 16). A computer-readable storage medium can be any medium that cantangibly contain or store computer-executable instructions for use by orin connection with the instruction execution system, apparatus, ordevice. In some examples, the storage medium is a transitorycomputer-readable storage medium. In some examples, the storage mediumis a non-transitory computer-readable storage medium. The non-transitorycomputer-readable storage medium can include, but is not limited to,magnetic, optical, and/or semiconductor storages. Examples of suchstorage include magnetic disks, optical discs based on CD, DVD, orBlu-ray technologies, as well as persistent solid-state memory such asflash, solid-state drives, and the like. Personal electronic device 580is not limited to the components and configuration of FIG. 5J, but caninclude other or additional components in multiple configurations.

Attention is now directed towards embodiments of user interfaces (“UI”)and associated processes that are implemented on an electronic device,such as portable multifunction device 100, device 300, or device 500.

FIG. 6 includes diagram 600, illustrating a physical structure and anexemplary set of devices in accordance with some embodiments. FIG. 6 isused to illustrate the processes described below, including theprocesses in FIGS. 8A-8B, FIG. 10, and FIG. 12.

Diagram 600 includes home 602 and a set of devices (e.g., device 646,device 604, etc.). Home 602 includes multiple rooms, such as kitchen 602a, dining room 602 b, bedroom 602 c, living room 602 d, and sunroom 602e. Kitchen 602 a is in the upper left portion of home 602 and diningroom 602 b is between kitchen 602 a and bedroom 602 c, where bedroom 602c is to the right of dining room 602 b. Kitchen 602 a and dining room602 b do not have a wall between them while dining room 602 b andbedroom 602 c have a wall between them. Living room 602 d is in thebottom left portion of home 602 and sunroom 602 e is in the bottom rightportion of home 602. In some embodiments, home 602 includes other roomsthan those depicted in FIG. 6 and/or excludes rooms that are depicted inFIG. 6. While diagram 600 depicts a home, it should be recognized thatthis is merely an example and techniques described herein can work withother types of physical structures, such as an office building, a hotel,an apartment, etc.

As illustrated in FIG. 6, each room in home 602 has at least one device.However, in some embodiments, a room in home 602 has no devices.

As illustrated in FIG. 6, John's phone 604 is in kitchen 602 a, lying ona counter between a stove and a sink. In dining room 602 b, dining roomspeaker 616 is sitting on a dining room table. In bedroom 602 c,personal tablet 634 is lying on a bed, bedroom speaker 636 is sitting ona nightstand that is adjacent to the bed, and bedroom light 632 ispositioned on the opposite side of the bed. In living room 602 d, livingroom speaker 646 and living room speaker 648 are on opposite sides of atelevision, and living room lamp 642 is positioned in front of thetelevision screen. In sunroom 602 e, sunroom speaker 656 and sunroomspeaker 658 are positioned on opposite sides of the room.

As illustrated in FIG. 6, computer device 674 and Jane's phone 684 areoutside of home 602. Computer device 674 is position within thedashboard of a car that is distance 670 from home 602; and Jane's phone684 is distance 680 away from home 602 and is currently in thepossession of Jane. In some embodiments, computer device 674 isconnected to an electronic device, such as a cellphone (e.g., a personalphone that is similar to John's phone 604 and/or Jane's phone 684), andthe electronic device provides instructions to computer device 674 thatcauses the computer device 674 to display visual content. Distance 670and 680 are more than a threshold distance away (e.g., 1, 5, 10 meters)from the home, such that computer device 674 and Jane's phone 684 arenot considered to be in locations that correspond to home 602 (e.g., arenot within a geofence of the home or are not connected to a wirelessnetwork associated with (e.g., set up within) home 602)).

In this example, the speakers (e.g., dining room speaker 616, bedroomspeaker 636, living room speaker 646, living room speaker 648, sunroomspeaker 656, and sunroom speaker 658), lights (e.g., bedroom light 632and living room lamp 642), and personal devices (e.g., John's phone 604,tablet 634, computer device 674, and Jane's phone 684) shown in FIG. 6are assigned to (e.g., programmatically mapped (e.g., by a user accountassociated with home 602, such as John's user account or Jane's useraccount) to a group that corresponds to) home 602.

In addition, each individual speaker and light are also assigned to(e.g., mapped to a group that corresponds to) a room in which it ispositioned. For example, dining room speaker 616 is assigned to (e.g.,programmatically mapped to a group that corresponds to) dining room 602b; bedroom speaker 636 and bedroom light 632 are assigned to bedroom 602c; living room lamp 642, living room speaker 646, and living roomspeaker 648 are assigned to living room 602 d; and sunroom speaker 656and sunroom speaker 658 assigned to sunroom 602 e. In some embodiments,each group that corresponds to each room is also assigned to home 602.Thus, devices that are mapped to a room of home 602 are alsoconcurrently mapped to home 602.

An external device cannot be assigned to two different groups of devicesthat correspond to different rooms. For example, dining room speaker 616cannot be assigned to kitchen 602 a (e.g., a group corresponding tokitchen 602 a) and dining room 602 b at the same time. However, twogroups of devices that are each assigned to respective rooms can bemapped to a (same) zone. In this example, entertainment zone 660 isassigned to living room 602 d in FIG. 6 and sunroom 602 e in FIG. 6. Inthis example, entertainment zone 660 is not a physical room in home 602c; however, the entertainment zone is programmatically defined as beingassigned to sunroom living room 602 d and sunroom 602 e. In someembodiments, two different zones share a group of devices (e.g., includesome overlapping devices and each include some non-overlapping devices).For example, the group of devices that are assigned to sunroom 602 e canbe mapped to entertainment zone 660 and an eating zone (e.g., where theeating zone is assigned to a group of devices that are assigned todining room 602 b and sunroom 602 e).

In this example, the personal devices (e.g., John's phone 604, tablet634, and Jane's phone 684) are not programmatically mapped to a groupthat corresponds to a room and only are mapped to a group thatcorresponds to a home.

In some embodiments, the speakers, lights, and personal devices (e.g.,John's phone 604, tablet 634, Jane's phone 684) shown in FIG. 6 aresmart devices that are directly connected to each other or indirectlyconnected to each other via one or more networks (e.g., wirelessnetworks (e.g., Bluetooth, NFC, Wi-Fi, 4G, etc.)). In some embodiments,other smart devices, such as smart appliances, smart thermostats, smartplug outlets, etc. are connected to the one or more networks. In someembodiments, the one or more personal devices can also include othertypes of personal devices, such as smartwatches, laptops, desktops, etc.

FIGS. 7A-7AH illustrate exemplary electronic devices and user interfacesfor sending an audio message in accordance with some embodiments. Theuser interfaces in these figures are used to illustrate the processesdescribed below, including the processes in FIGS. 8A-8B.

FIG. 7A illustrates electronic device 700 displaying user interface 702.In FIG. 7A, electronic device 700 is a portable multifunction device andhas one or more components described above in relation to one or more ofdevices 100, 200, 300, and 500. With respect to the figures below, insome embodiments, device 700 is operating at a location that is lessthan a predetermined distance away from home 602; however, in otherembodiments, device 700 is operating at a location that is greater thana predetermined distance away from home 602. In some embodiments, device700 is connected to a network (e.g., wireless network) that isassociated with home 602, such that device 700 is connected to the samewireless network to which at least one the devices (e.g., speakers,lights) located in home 602 are connected. In some embodiments, morethan one wireless network is associated with home 602. In someembodiments, device 700 is connected to at least one of those networks.In some embodiments, device 700 is not connected to the one or morenetworks that is associated with home 602. Thus, device 700 can sendaudio multicast messages and communicate with devices within home 602,irrespective of the distance that device 700 is from home 602 or thenetwork to which device 700 is connected.

While this description often refers to messages as multicast messages(or audio multicast messages), it should be recognized that suchmessages are optionally sent using (1) a multicast communication, (2) abroadcast communication, (3) a unicast communication, (4) or anycombination thereof (e.g., (1) a first device (e.g., a controller) sendsa unicast communication to a second device (e.g., a hub in a home) and(2) the second device sends either a multicast or a broadcastcommunication (e.g., including content from the unicast communication)to one or more third devices (e.g., one or more accessory devices withinthe home and/or one or more personal devices). In some embodiments, amulticast communication is a communication sent to multiple receipts. Insome embodiments, a broadcast communication is a communicationtransmitted to devices on a channel or network. In some embodiments, abroadcast communication is a communication transmitted to all devices ona channel or network. In some embodiments, a unicast communication is acommunication transmitted to a single recipient. In such embodiments,multiple unicast communications are sent when content needs to be sentto multiple devices.

In some embodiments, a broadcast communication is a communicationtransmitted to all devices on a channel or network. In such embodiments,multiple unicast communications are sent when content needs to be sentto multiple devices.

As illustrated in FIG. 7A, user interface 702 includes applicationsicons 704, such as home application icon 704 a. Each of applicationicons 704 corresponds to an application, such that, when an applicationicon is selected, the selection will cause device 700 to display a userinterface of a respective application. For example, home applicationicon 704 a corresponds to a home application. At FIG. 7A, device 700detects a tap gesture 750 a (e.g., a selection) at a location thatcorresponds to home application icon 704 a.

As illustrated in FIG. 7B, in response to detecting tap gesture 750 a,device 700 displays user interface 710 of the home application. Userinterface 710 is a user interface for controlling multiple externaldevices assigned to home 602. User interface 710 includes home locationidentifier 712, status information 714, macro controls 716, externaldevice controls 718, navigations controls 722, home multicastinstruction 724, and home multicast affordance 726 (e.g., selectableuser interface object).

Home location identifier 712 is a name that corresponds to home 602. Inthis example, the location identifier is part of the address (e.g., “123Main Street”) of home 602. In some embodiments, the location identifieris an alias, such as an alias assigned to the home. In some embodiments,the alias is a system-created alias. In some embodiments, the alias is auser-created alias. In some embodiments, the location identifier is setby a user of device 700 or an external device. In some embodiments, thelocation identifier is a graphical user interface object, such as animage.

Status information 714 includes information concerning the currentstatus of some of the devices assigned to home 602. Here, statusinformation 714 indicates that at least one light is on in the home 602.In some embodiments, status information 714 includes informationconcerning the status of multiple devices (e.g., “bedroom fan on”).

As illustrated in FIG. 7B, there are two rows of macro controls 716below home location identifier 712 and status information 714. Macrocontrols 716 are affordances (e.g., selectable user interface objects)for initiating a respective macro. In some embodiments, the respectivemacro causes one or more externals devices to perform an action. Forexample, the “Good Night” macro control, when selected, cause one ormore lights to turn off, one or more speakers to stop playback media,and/or one or more deadbolt locks to lock.

Additionally, in FIG. 7B, user interface 710 includes rows for externaldevice controls 718 below macro controls 716. In FIG. 7B, the two rowsof external device controls 718 extend across three columns, so there isa total of six displayed (e.g., where three device controls are on eachrespective row). In some embodiments, a user may have fewer externaldevices to control requiring fewer external device controls 718. Inother embodiments, a user may have more external devices to controlrequiring more external device controls 718. In some embodiments, whenthere are more external device controls 718 to display than screen spaceavailable, the screen scrolls (e.g., up and down or left and right) todisplay all external device controls. External device controls 718include dining room speaker control 718 a, living room lamp control 718b, living room speakers control 718 c, sunroom speakers control 718 d,bedroom speaker control 718 e, and bedroom light control 718 fAdditionally, each of external device controls 718 include statusinformation that is representative of the current state of eachrespective external device. For example, the status information fordining room speaker control 718 a indicates that dining room speaker 616is currently playing media (which is not represented by statusinformation 714 in this example) and the status information for livingroom lamp control 718 b indicates that living room lamp 642 is currentlyoff (which is represented by status information 714 in this example).

Each of external device controls 718, when selected, control at leastone device assigned to home 602. For example, dining room speakercontrol 718 a, when selected, causes dining room speaker 616 to play orpause music. In some embodiments, when dining room speaker control 718 ais selected, the status information on dining room speaker control 718 achanges to show that dining room speaker 616 has paused the playback ofmedia (e.g., as shown by bedroom speaker control 718 e) or stopped theplayback of media. Some of external device controls 718 control multipledevices when selected. For example, sunroom speakers control 718 d, whenselected, causes sunroom speakers 656 and 658 to play or pause theplayback of media and the status information displayed on sunroomspeakers control 718 d changes accordingly. Some of external devicecontrols 718 are not speaker device controls. For example, externaldevice controls 718 includes living room lamp control 718 b that, whenselected, causes living room lamp 642 to turn on/off and the statusinformation displayed on living room lamp control 718 b changesaccordingly.

In some embodiments, external device controls 718 are selected from agroup of other external controls that are also assigned to home 602. Forexample, the six controls that are shown in FIG. 7B can be selected froma group of 10, 20, or 50 (or any other number greater than 6) externalcontrols that are also assigned to home 602. In some embodiments,external device controls 718 are selected based on a determination thateach of these external device controls are a favorite control. In someembodiments, the determination of whether an external device is afavorite control is based on whether the external device control (or theexternal device) was most recently used, most frequently used, newlyadded, designated as a favorite, important external device control(e.g., via the detection of one or more inputs), etc., or anycombination thereof.

At the bottom of user interface 710 are navigation controls 722.Navigation controls 722 include home navigation control 722 a, roomnavigation control 722 b, and automation navigation control 722 c. Eachof navigation controls 722, when selected, causes device 700 to displaya different user interface for the home application.

As illustrated in FIG. 7B, user interface 710 includes home multicastinstruction 724 that is adjacent to home multicast affordance 726. InFIG. 7B, home multicast instruction 724 indicates that a user shouldselect (e.g., “tap”) home multicast affordance 726 in order to send anaudio message that will be multicast to one or more devices assigned tohome 602. At FIG. 7B, device 700 detects tap gesture 750 b at a locationthat corresponds to home multicast affordance 726.

In response to detecting tap gesture 750 b, device 700 initiates aprocess for sending an audio multicast message to one or more devicesassigned to home 602. In some embodiments, the process includesdisplaying user interfaces depicted in FIGS. 7C, 7D, and 7E, asdescribed below. In some embodiments, the process includes displayinguser interface depicted in FIG. 7C1, as described below.

As illustrated in FIG. 7C, in response to detecting tap gesture 750 b,user interface 730 includes home destination identifier 732, exitaffordance 734, and multicast recording affordance 736. Home destinationidentifier 732 indicates that the audio multicast message is directed toa group of devices assigned to home 602 (e.g., will receive an audiomulticast message). Exit affordance 734 is an affordance for exiting outof user interface 730. In some embodiments, when exit affordance 734 isselected, device 700 ceases to display user interface 730 andre-displays user interface 710, as depicted in FIG. 7B. At FIG. 7C,device 700 detects tap gesture 750 c at a location that corresponds tomulticast recording affordance 736.

As illustrated in FIG. 7D, in response to detecting tap gesture 750 c,device 700 displays user interface 740. User interface 740 includes homedestination identifier 732 and exit affordance 734, which were alsoincluded in user interface 730. In addition, user interface 740 alsoincludes audio status bar 742 and audio wave 744 a.

Additionally, at FIG. 7D, device 700 also starts recording an audiomulticast message in response to detecting tap gesture 750 c. The audiofor the audio multicast message is detected via one or more microphonesof device 700. The audio for the audio multicast message is detectedfrom one or more external sources (e.g., such as a user voice). Here,the recorded audio is represented by audio wave 744 a, which is fillingup audio status bar 742 over a period of time as more audio input isreceived by device 700.

As illustrated in FIG. 7D, in response to detecting tap gesture 750 c,device 700 displays multicast recording stop affordance 746 at theposition that multicast recording affordance 736 was previouslydisplayed in FIG. 7C. At FIG. 7D, device 700 detects tap gesture 750 dat a location that corresponds to multicast stop recording affordance746.

As illustrated in FIG. 7E, in response to detecting tap gesture 750 d,device 700 displays user interface 750. At FIG. 7E, in response todetecting tap gesture 750 d, device 700 has ceased recording orcapturing audio from the one or more external sources. As such, userinterface 750 includes audio status bar 742 that is mostly filled byaudio wave 744 a, which represents at least a portion of the recordedaudio multicast message. As illustrated in FIG. 7E, user interface 750includes audio multicast message playback affordance 752 and audiomulticast message deletion affordance 754 (e.g., in addition toincluding home destination identifier 732, exit affordance 734, audiostatus bar 742, audio wave 744 a). In some embodiments, in response todetecting a selection of audio multicast message playback affordance752, device 700 will initiate playback of the audio multicast message.In some embodiments, in response to detecting a selection of audiomulticast message deletion affordance 754, device 700 will delete theaudio multicast message. In some embodiments, in response detecting aselection of audio multicast message deletion affordance 754, device 700re-displays user interfaces 710, 730, and/or 740.

As illustrated in FIG. 7E, in response to detecting tap gesture 750 d,device 700 displays multicast send affordance 756 at the position thatmulticast stop recording affordance 746 was previously displayed in FIG.7D (that is also the position in which multicast recording affordance736 was displayed in FIG. 7C).

As an alternative to the flow described in FIGS. 7C, 7D, and 7E, device700 can display the user interface illustrated in FIG. 7C1 in responseto detecting tap gesture 750 b in FIG. 7B. As illustrated in FIG. 7C1,in response to detecting tap gesture 750 b, device 700 displays userinterface 741 and automatically starts recording an audio multicastmessage (e.g., without detecting any further gestures, such as tapgesture 750 c). As illustrated in FIG. 7C1, user interface 741 includeshome destination identifier 732, exit affordance 734, audio wave 744 a,multicast send affordance 747, which device 700 displays using similartechniques to those described above in relation to FIGS. 7C, 7D, and 7E.For example, in FIG. 7C1, audio wave 744 a is dynamic and changes overtime with respect to one or more characteristics (e.g., volume level,pitch, tone) of audio that is currently being recorded over the lengthof time that audio is being record. In some embodiments, audio wave 744a moves to the left as audio is being recorded over the length of timethat audio is being record.

In some embodiments, audio is recorded using user interface 710 of FIG.7B or user interface 741 of FIG. 7C1, instead of through the flowdescribed in FIGS. 7C, 7D, and 7E and/or through the flow described inFIG. 7C. For example, in response to detecting a hold gesture on homemulticast affordance 726, device 700 starts recording an audio multicastmessage. In such an example, the recording is stopped when the holdgesture is terminated (e.g., released), causing replacement of homemulticast affordance 726 with a send affordance (e.g., similar tomulticast send affordance 756). In some embodiments, when the holdgesture is terminated, the audio is automatically sent as a multicastmessage. In some embodiments, when the hold gesture is terminated, homemulticast instruction 724 is replaced with an affordance to play theaudio (e.g., similar to audio status bar 742) and an affordance todelete the audio (e.g., similar to audio multicast message deletionaffordance 754). In another example, in response to detecting a tapgesture on home multicast affordance 726, home multicast affordance 726operates similar to multicast recording affordance 736 (e.g., replacedwith a stop affordance (similar to multicast recording stop affordance746)). In such an example, in response to detecting a tap gesture on thestop affordance, (1) the stop affordance is replaced with a sendaffordance (e.g., similar to multicast send affordance 756) and (2) homemulticast instruction 724 is replaced with an affordance to play theaudio (e.g., similar to audio status bar 742) and an affordance todelete the audio (e.g., similar to audio multicast message deletionaffordance 754). At FIG. 7E, device 700 detects tap gesture 750 e at alocation that corresponds to multicast send affordance 756, or device700, at FIG. 7C1, device detects tap gesture 750 c 1 at a location thatcorresponds to multicast send affordance 747.

As illustrated in FIG. 7F, in response to detecting tap gesture 750 e ortap gesture 750 c 1 (or a tap gesture on a send affordance, as describedin the previous paragraph), device 700 causes the audio multicastmessage to be multicast (sent) to a set of external devices that areassigned to home 602. Such sending optionally includes (1) sending amessage including the audio multicast message to another device (e.g., adevice located within the home) and (2) the other device sending theaudio multicast message to the set of external devices (in someembodiments, the other device identifies the set of external devicesinstead of device 700). Table 706 is provided to illustrate the externaldevices to which the audio multicast message is directed. For example, acheckmark in the “received audio multicast message” column indicatesthat the audio multicast message was directed to (e.g., sent to) theexternal device on the respective row of the checkmark and a dash in the“received audio multicast message” column indicates that the audiomulticast message was not directed to (e.g., not sent to) the externaldevice on the respective row of the dash.

In FIG. 7F, table 706 shows that the audio multicast message wasdirected to (and received by) each speaker (e.g., dining room speaker616, bedroom speaker 636, living room speaker 646, living room speaker648, sunroom speaker 656, and sunroom speaker 658). Notably, the audiomulticast message was also directed to the personal devices (e.g.,John's phone 604, tablet 634, and Jane's phone 684). In someembodiments, the audio multicast message is directed to the personaldevice irrespective of whether the personal device was at a locationthat corresponds to the home or not (e.g., independent of whetherdistances 670 and 680 exceed the threshold distance). For example, theaudio multicast message was directed to John's phone 604 and tablet 634,which in FIG. 6 are located inside of home 602. In addition, the audiomulticast message was also directed to computer device 674 and Jane'sphone 684, which are outside of home 602 (respectively distance 670 anddistance 680 away from home 602). In some embodiments, the audiomulticast message is directed to the personal device dependent onwhether the personal device is included in a network of the home (e.g.,the audio multicast message is directed to the personal device when thepersonal device is included in the network of the home).

Table 706 in FIG. 7F also indicates that the audio multicast message wasnot directed to some external devices (and thus did not received theaudio multicast message), although they are assigned to home 602. Forexample, table 706 shows that the audio multicast message was notdirected to bedroom light 632 and living room lamp 642. In someembodiments, audio multicast messages are not directed to some types ofexternal devices, irrespective of whether the external devices areassigned to a group of devices to which the audio multicast message isbeing sent. In some embodiments, these types of external devices includedevices that do not have one or more audio input devices (e.g.,microphones) and/or one or more audio output devices (e.g., speakers).In some embodiments, these types of external devices include devicesthat are inactive, not powered, or not functional at the time (or beforethe time) that the audio message is sent.

As illustrated in FIG. 7F, at some point after detecting tap gesture 750e or tap gesture 750 c 1 (or a tap gesture on a send affordance, asdescribed above), device 700 re-displays user interface 710. In someembodiments, user interface 710 is displayed without device 700detecting any additional inputs after detecting tap gesture 750 e or tapgesture 750 c 1 (e.g., after a predetermined period of time afterdetecting tap gesture 750 e or tap gesture 750 c 1) (or a tap gesture ona send affordance, as described above (e.g., after a predeterminedperiod of time after detecting the tap gesture)). In some embodiments,user interface 710 is displayed when one or more additional inputs aredetected, such as a tap gesture input that is detected on exitaffordance 734 in FIG. 7E.

In some embodiments, in response to detecting tap gesture 750 f 1 at alocation that corresponds to living room lamp control 718 b, device 700will send instructions that causes living room lamp 642 to turn on (oroff if current state was on). In some embodiments, in response todetecting tap gesture 750 f 2 at a location that corresponds to livingroom speaker control 718 c, device 700 will send instructions that causeliving room speakers 646 and 648 (or at least one of them) to initiateplayback of media (e.g., or pause playback of media if current state wasplaying). At FIG. 7F, device 700 detects tap gesture 750 f 3 at alocation that corresponds to room navigation affordance 722 b.

As illustrated in FIG. 7G, in response to detecting tap gesture 750 f 3,device 700 displays user interface 760 of home application and indicatesthat room navigation control 722 b is selected (e.g., bold in FIG. 7G)instead of home navigation control 722 a (e.g., not bold in FIG. 7G).User interface 760 is a user interface for controlling multiple externaldevices assigned to living room 602 d, as indicated by living roomlocation identifier 762. User interface 710 includes macro controls 766,external device controls 768, navigation controls 722, living roommulticast instruction 774, and living room multicast affordance 776.These elements of user interface 760 are specific controls for devicesof the living room as opposed to the elements of user interface 710 thatwere controls for devices of the entire home. In some embodiments, oneor more of these elements of user interface 760 have a similar purposeor functionality to one or more of the elements of user interface 710described above (e.g., in FIG. 7B). In some embodiments, user interface760 includes status information that includes the current status ofdevices that are assigned to living room 602 d. In some embodiments,device 700 displays user interface 760 of home application afterreceiving a swipe gesture on user interface 710.

Notably, the user interfaces (e.g., user interfaces 710 and 760) in thehome application are laid out in a hierarchy. For example, userinterface 710 is the top node of the hierarchy and includes controls fora group of external devices that are assigned to home 602. And, whiledisplaying user interface 710, input (e.g., a single input (e.g., tapgesture 750 f)) can be detected that causes device 700 to display userinterface 760, which is a lower node in the hierarchy and includescontrols for a group of external devices that are assigned to arespective room in home 602 (e.g., living room 602 d) (and not otherrooms).

As illustrated in FIG. 7G, external device controls 768 include livingroom speakers control 768 a, living room lamp control 768 b, living roomswitch (e.g. light switch) control 768 c, and living television control768 d. In FIG. 7G, living room speakers control 768 a and living roomlamp control 768 b causes each of their respective external device(s) toperform the same action(s), when selected, than the action(s) thatliving room lamp device control 718 b and living room speakers devicecontrol 718 c would cause their respective external device(s) to perform(e.g., in FIG. 7B). Thus, in this example, living room speakers control768 a and living room lamp control 768 b are effectively the samecontrols as living room speakers device control 718 c and living roomlamp device control 718 b, although they look visually different fromeach of their respective counterparts. In some embodiments, living roomspeakers control 768 a and living room lamp control 768 b do not lookvisually different from living room speakers device control 718 c andliving room lamp device control 718 b.

When compared to external device controls 718 displayed in FIG. 7B,external device controls 768 include at least one external devicecontrol (e.g., living room speakers control 768 a and living room lampcontrol 768 b) that was displayed as a part of external device controls718 in FIG. 7B and at least one external device control (e.g., livingroom switch control 768 c and living room television controls 768 d)that was not. In other words, at least one external device (e.g., livingroom speakers 646 and 648) can be controlled from user interfaces 710and 760 and at least one external device (e.g., bedroom speaker 636 andliving room television in user interface 760) can only be controlledfrom only one of user interface 710 and user interface 760. In someembodiments, in response to detecting tap gesture 750 g 1 at a locationthat corresponds to living room lamp control 768 b, device 700 will sendinstructions that causes living room lamp 642 to turn on (or off ifcurrent state was on). In some embodiments, in response detecting tapgesture 750 g 2 at a location that corresponds to living room speakercontrol 768 a, device 700 will send instructions that causes living roomspeakers 646 and 648 (or at least one of them) to initiate playback ofmedia (e.g., or pause playback of media if current state was playing).

Additionally, while external device controls 718 were designated asbeing a favorite set of controls for home 602 in FIG. 7B, externaldevice controls 768 are a set of all the controls that are assigned toliving room 602 d in FIG. 7G. In FIG. 7B, user interface 760 does notinclude any external devices that are not assigned to living room 602 d.For example, dining room speaker device control 718 a, sunroom speakersdevice control 718 d, bedroom speaker device control 718 e, and bedroomlight device control 718 f are not displayed on user interface 760. Insome embodiments, a user interface for controlling multiple externaldevices for another room (e.g., such as the bedroom, sunroom, kitchen)is displayed in response to detecting tap gesture 750 f 3. In someembodiments, a user for controlling multiple external devices foranother room is displayed in response to one or more inputs (e.g., aswipe input) that is detected at FIG. 7G.

As illustrated in FIG. 7G, user interface 760 includes living roommulticast instruction 774 that is adjacent to living room multicastaffordance 776. In FIG. 7G, living room multicast instruction 774indicates that a user should select (e.g., “tap”) living room multicastaffordance 776 in order to send an audio multicast message that will bemulticast to one or more devices assigned to living room 602 d. Here,living room multicast affordance 776 does not look visually differentfrom home multicast affordance 726 (e.g., in FIG. 7B) and is displayedin the same position on device 700 that home multicast affordance 726was displayed in FIG. 7B. In some embodiments, living room multicastaffordance 776 looks visually different from home multicast affordance726. At FIG. 7G, device 700 detects tap gesture 750 g 3 at a locationthat corresponds to living room multicast affordance 776.

In response to detecting tap gesture 750 g 3, device 700 initiates aprocess for sending an audio multicast message to one or more devicesassigned to living room 602 d. In some embodiments, the process includesdisplaying user interface 780 a, as depicted in FIG. 7H. In someembodiments, the process includes device 700 displaying a userinterface, similar to user interface 741 of FIG. 7C1, that is displayedin response to detecting tap gesture 750 g 3. In some embodiments, theprocess includes device 700 displaying a user interface, similar to userinterface 741 of FIG. 7C1, to record an audio multicast message to sendto a group of devices that are assigned to living room 602 d. In someembodiments, the process includes device 700 displaying user interfaces,as described below in FIGS. 7H-7J, where audio is recorded and sent to agroup of devices that assigned to living room 602 d.

As illustrated in FIG. 7H, in response to detecting tap gesture 750 g 3,user interface 780 a is similar to user interface 730 in FIG. 7C, suchthat user interface 780 a also includes exit affordance 734 andmulticast recording affordance 736. However, user interface 780 aincludes living room destination identifier 782 instead of homedestination identifier 732. Living room destination identifier 782indicates that a group of devices that are assigned to living room 602 dwill receive a multicast message sent via user interface 780 a. In someembodiments, user interface 780 a does not list the individualnames/identifiers of the devices that are part of the group. In someembodiments, one or more techniques discussed above in relation to userinterface 730 can be applied to user interface 780 a.

Turning to FIG. 7K, in some embodiments, user interface 780 b isdisplayed in response to detecting tap gesture 750 g 3 and in accordancewith a determination that one or more (or all) external devices (e.g.,speakers) assigned to living room 602 d are not available (e.g., areinactive or off). As illustrated in FIG. 7K, user interface 780 bincludes an unavailable indication 788 to indicate that one or more ofthe external devices assigned to living room 602 d are unavailable (orall and not when less than all are unavailable) and multicast recordingaffordance 736 is displayed as being dimmed or greyed-out. At FIG. 7K,in response to detecting tap gesture 750 k, device 700 continues todisplay user interface 780 b (e.g., until the one or more externaldevices assigned to the living room 602 d are available) becausemulticast recording affordance 736 is disabled. In some embodiments, asimilar user interface to user interface 780 b is displayed in responseto device 700 detecting tap gesture 750 c in FIG. 7C.

Turning back to FIG. 7H, device 700 receives tap gesture 750 h at alocation that corresponds to multicast recording affordance 736.

As illustrated in FIG. 7I, in response to receiving tap gesture 750 h,device 700 starts recording audio (e.g., using a microphone of device700) from one or more external sources and displays user interface 784,which is similar to user interface 740 in FIG. 7D. Like user interface740, user interface 784 includes exit affordance 734, audio status bar742, and multicast recording stop affordance 746. However, userinterface 784 includes audio wave 744 b that represents different audiothat is being record from the audio recorded in FIG. 7D that wasrepresented by audio wave 744 a. User interface 784 also includes livingroom destination identifier 782 instead of home destination identifier732. In some embodiments, one or more techniques discussed above inrelation to user interface 740 can be applied to user interface 784. AtFIG. 7I, device 700 detects tap gesture 750 i at a location thatcorresponds to multicast stop recording affordance 746.

As illustrated in FIG. 7J, in response to detecting tap gesture 750 i,device 700 displays user interface 786 and ceasing recording or captureaudio from one or more external sources. User interface 786 is similarto user interface 750, except for audio wave 744 b being displayed(instead of audio wave 744 a) and living room destination identifier 782being displayed (instead of home destination identifier 732). In someembodiments, one or more techniques discussed above in relation to userinterface 750 can be applied to user interface 786. At FIG. 7J, device700 detects tap gesture 750 j at a location that corresponds tomulticast send affordance 756 (or a tap gesture (e.g., similar to tapgesture 750 c 1) on a user interface that is similar to user interface741 of FIG. 7C1 as described above).

In some embodiments, device 700 displays user interface 790 in responseto detecting tap gesture 750 j (or a tap gesture (e.g., similar to tapgesture 750 c 1) on a user interface that is similar to user interface741 of FIG. 7C1 as described above), as illustrated in FIG. 7L. Device700 displays user interface 790 when a determination that the audiomulticast message cannot be deliver to one or more (or all) of theexternal devices assigned to living room 602 d. User interface 790includes not delivered indication 791 to indicate that the audiomulticast message was not delivered to the targeted external devices. Insome embodiments, a similar user interface to user interface 790 isdisplayed in response to device 700 detecting tap gesture 750 e in FIG.7E or tap gesture 750 c 1 in FIG. 7C1.

In some embodiments, one or more of user interface 730, 740, 750, 780 a,780 b, 784, 786, and 790 include an affordance to swap the intendedtarget of the recorded audio multicast message (e.g., change thedestination identifier). For example, after initiating a process forsending an audio multicast message to a first group of device (e.g.,devices assigned to home 602) (such as while showing FIG. 7J and beforereceiving input 750 j or a tap gesture (e.g., similar to tap gesture 750c 1) on a user interface that is similar to user interface 741 of FIG.7C1 as described above), device 700 can detect a selection of theaffordance to swap the intended target of the recorded audio multicastmessage (e.g., or multiple affordances to swap the intended target) fromthe first group of devices to a second group of device (e.g., devicesassigned to dining room 602 b).

In some embodiments, audio is recorded and sent from a user interfacesimilar to user interface 741 of FIG. 7C1 that is displayed in responseto detecting tap gesture 750 g. In some embodiments, audio is recordedfrom user interface 760 in FIG. 7G, instead of through the flowdescribed in FIGS. 7H-7J. For example, in response to detecting a holdgesture on living room multicast affordance 776, device 700 startsrecording an audio multicast message. In such an example, the recordingis stopped when the hold gesture is terminated (e.g., released), causingreplacement of living room multicast affordance 776 with a sendaffordance (e.g., similar to multicast send affordance 756). In such anexample, living room multicast instruction 774 is replaced with anaffordance to play the audio (e.g., similar to audio status bar 742) andan affordance to delete the audio (e.g., similar to audio multicastmessage deletion affordance 754). In another example, in response todetecting a tap gesture on living room multicast affordance 776, livingroom multicast affordance 776 operates similar to multicast recordingaffordance 736 (e.g., replaced with a stop affordance (similar tomulticast recording stop affordance 746)). In such an example, inresponse to detecting a tap gesture on the stop affordance, the stopaffordance is replaced with a send affordance (e.g., similar tomulticast send affordance 756) and living room multicast instruction 774is replaced with an affordance to play the audio (e.g., similar to audiostatus bar 742) and an affordance to delete the audio (e.g., similar toaudio multicast message deletion affordance 754).

As illustrated in Table 706 of FIG. 7M, in response to detecting tapgesture 750 j (or a tap gesture on a send affordance, as described inthe previous paragraph or a tap gesture (e.g., similar to tap gesture750 c 1) on a user interface that is similar to user interface 741 ofFIG. 7C1 as described above), device 700 causes the audio multicastmessage to be multicast (sent or directed to) to a set of externaldevices that are assigned to living room 602 d. Such sending optionallyincludes (1) sending a message including the audio multicast message toanother device (e.g., a device located with the living room) and (2) theother device sending the audio multicast message to the set of externaldevices (in some embodiments, the other device identifies the set ofexternal devices instead of device 700). As shown by table 706, theaudio multicast message is direct to only living room speakers 646 and648, which receive the audio multicast message. Notably, the audiomulticast message is not directed to the personal devices (e.g., John'sphone 604, tablet 634, and Jane's phone 684) because personal devicesare not assigned to groups that correspond to rooms (e.g., personaldevices are not assigned to the group that corresponds to living room602 d). Additionally, the audio multicast message is not directed toliving room lamp 642 because living room lamp 642 is not the type ofexternal device that can receive audio multicast messages for similarreasons discussed above (e.g., in FIG. 7F).

At FIG. 7M, device 700 detects voice input 792 a that includes thephrase “HEY ASSISTANT, MULTICAST HELLO.” Sometime after detecting aportion of voice input 792 a, device 700 displays user interface 794with indication 794 a (e.g., on the left side of user interface 794).Here, indication 794 a is a response the portion of voice input 792 a.Indication 794 a is text that reads: “WHAT CAN I HELP YOU WITH?” In someembodiments, device 700 outputs media that includes an audible versionof indication 794 a.

At FIG. 7N, after displaying user interface 794 in FIG. 7M, device 700displays indication 794 b on the right side of user interface 794. Inaddition, device 700 displays indication 794 c on the left side of userinterface 794 and below indication 794 b. Indication 794 b is text thatreads “HEY ASSISTANT, MULTICAST HELLO” which is the text thatcorresponds to the audible phrase included in voice input 792 a.Additionally, indication 794 c is text that reads “OK, I'VE SENT YOURMULTICAST TO 123 MAIN ST.” Notably, 123 Main St. is the identifier forhome 602 (e.g., home location identifier 712 in FIG. 7B). In someembodiments, device 700 outputs media that includes an audible versionof response 794 c.

As shown by table 706 in FIG. 7N, after detecting voice input 792 a,device 700 directs (e.g., sends) an audio multicast message to one ormore devices assigned to home 602, as described above in relation toFIG. 7F. Table 706 shows that the audio multicast message (e.g.,“Hello”) is directed to each speaker (e.g., dining room speaker 616,bedroom speaker 636, living room speaker 646, living room speaker 648,sunroom speaker 656, and sunroom speaker 658) of the home and that theaudio multicast message (e.g., “Hello”) is directed to the personaldevices (e.g., John's phone 604, tablet 634, Jane's phone 684) of thehome, irrespective of whether a device was at a location thatcorresponds to the home or not and/or irrespective of whether the deviceis connected to the network associated with the home (as discussedabove).

As illustrated in FIG. 7O, sometime after displaying user interface 794in FIG. 7N, device 700 displays user interface 710. While displayinguser interface 710, device 700 detects gesture 750 o (e.g., a long pressgesture) at a location that corresponds to bedroom speaker control 718 eat FIG. 7O.

As illustrated in FIG. 7P, in response to detecting gesture 750 o,device 700 displays user interface 796 that includes a plurality ofsetting affordances for controlling bedroom speaker 636 (e.g., a roomsetting (e.g., where bedroom is selected in FIG. 7P to show that bedroomspeaker 636 is assigned to bedroom 602 c), a favorites setting (e.g.,where include in favorites is selected in FIG. 7P, which can causebedroom speaker control 718 e to be displayed on user interface 710)).As illustrated in FIG. 7P, one of the settings affordances is multicastsetting affordance 796 a, which includes an indication that bedroomspeaker 636 is configured to receive multicast messages. At FIG. 7P,device 700 detects tap gesture 750 p at a location that corresponds tomulticast setting affordance 796 a.

As illustrated in FIG. 7Q, in response to detecting tap gesture 750 p,device 700 displays user interface 798, which includes multicastsettings toggle 798 a. In FIG. 7Q, multicast settings toggle 798 aindicates that bedroom speaker 636 is configured to receive multicastmessages (e.g., circle displayed to the right side of the settingstoggle). At FIG. 7Q, device 700 detects tap gesture 750 q at a locationthat corresponds to multicast setting affordance 798 a.

As illustrated in FIG. 7R, in response to detecting tap gesture 750 q,device 700 changes the state of multicast toggle 798 a (e.g., movescircle to the left side of the settings toggle) to indicate that bedroomspeaker 636 is not configured to receive multicast messages. In someembodiments, multicast setting affordance 796 a in FIG. 7P is selectableto change a setting from on to off without navigating to a differentuser interface (e.g., user interface 798).

At FIG. 7S, device 700 detects voice input 792 b that includes thephrase “HEY ASSISTANT, MULTICAST TO THE SUNROOM, HELLO.” Sometime afterdetecting a portion of voice input 792 b, device 700 displays userinterface 794 with indication 794 d (e.g., on the left side of userinterface 794). Here, indication 794 d is a response the portion ofvoice input 792 b. Indication 794 d is text that reads “WHAT CAN I HELPYOU WITH?” In some embodiments, device 700 outputs media that includesan audible version of indication 794 d.

At FIG. 7T, after displaying user interface 794 in FIG. 7S, device 700displays indication 794 e on the right side of user interface 794 anddisplays indication 794 f on the left side of user interface 794 (andbelow indication 794 e). Indication 794 e is text that reads, “HEYASSISTANT, MULTICAST TO THE SUNROOM, HELLO,” which is the text thatcorresponds to the audible phrase included in voice input 792 b.Additionally, indication 794 f is text that reads “OK, I'VE SENT YOURMULTICAST TO THE SUNROOM.” Notably, “sunroom” is an identifier for agroup of devices that are assigned to sunroom 602 e in FIG. 6.

As shown by table 706 in FIG. 7T, after detecting voice input 792 b,device 700 directs (e.g., sends) an audio multicast message to one ormore devices assigned to sunroom 602 e. Table 706 shows that the audiomulticast message (e.g., “Hello”) is directed to sunroom speaker 656 andsunroom speaker 658 and the audio multicast message (e.g., “Hello”) isnot directed to the other devices. Notably, voice input 792 b includedan identifier (e.g., “sunroom”) while voice 792 a did not include anidentifier. Thus, in some embodiments, when voice input is detectedwithout an identifier, device 700 will send the audio multicast messagesto the group of devices assigned to a default group. In someembodiments, the default group is a group that corresponds to a home(e.g., such has home 602), as described above in relation to FIG. 7N.

At FIG. 7U, device 700 detects voice input 792 c that includes thephrase “HEY ASSISTANT, MULTICAST TO THE ENTERTAINMENT ZONE, HELLO.”Sometime after detecting a portion of voice input 792 c, device 700displays user interface 794 with indication 794 g (e.g., on the leftside of user interface 794). Here, indication 794 g is a response to aportion of voice input 792 c. Indication 794 g is text that reads, “WHATCAN I HELP YOU WITH?” In some embodiments, device 700 outputs media thatincludes an audible version of indication 794 g.

At FIG. 7V, sometime after displaying user interface 794 in FIG. 7U,device 700 displays indication 794 i on the left side of user interface794 (and below indication 794 h). Indication 794 i is text that reads,“OK, I'VE SENT YOUR MULTICAST TO THE ENTERTAINMENT ZONE.” A zone is anidentifier for multiple groups of devices. Here, entertainment zone 660is assigned to the group of devices that corresponds to sunroom 602 e inFIG. 6 and the group of devices that corresponds to living room 602 d inFIG. 6.

As shown by table 706 in FIG. 7V, after detecting voice input 792 c,device 700 directs (e.g., sends) an audio multicast message to one ormore devices assigned to the entertainment zone. Table 706 shows thatthe audio multicast message (e.g., “Hello”) is directed to living roomspeaker 646, living room speaker 648, sunroom speaker 656, and sunroomspeaker 658 and that the audio multicast message (e.g., “Hello”) is notdirected to the other devices.

As illustrated in FIG. 7V, device 700 also displays multicastnotification 708 a, which indicates that a new audio multicast messagehas been received from the group of devices assigned to sunroom 602 e.The new audio multicast message is a reply to the audio multicastmessage sent in FIG. 7T.

At FIG. 7W, device 700 detects voice input 792 d that includes thephrase “Hey Assistant, Multicast, we are going to leave soon.”

At FIG. 7X, sometime after detecting voce input 792 d, device 700displays indication 794 j on the left side of user interface 794 andunder indication 794 i. Indication 794 j is text that reads: “HEYASSISTANT, MULTICAST, WE ARE GOING TO LEAVE SOON.” This text correspondsto the audible phrase included in voice input 792 d. Here, voice input792 d does not include an identifier. Additionally, at FIG. 7X, device700 also displays indication 794 k on the right side of user interface794 and under indication 794 j. Indication 794 k is text that reads:“OK, I'VE SENT YOUR MULTICAST TO THE SUNROOM.”

As shown by table 706 in FIG. 7X, after detecting voice input 792 d,device 700 initiates a process for sending the audio multicast message(e.g., “We are going to leave soon”) to the group of devices assigned tosunroom. In particular, device 700 directs (e.g., sends) the audiomulticast message to one or more devices assigned to sunroom 602 ewithout voice input 792 d including an identifier for the sunroom. Thus,because voice input 792 d was detected within a predetermined period oftime after receiving multicast notification 708 a, device 700intelligently directs the next detected audio multicast message to thesunroom instead of the default group (group corresponding to home 602).In some embodiments, device 700 intelligently routes the next detectedaudio multicast message to the sunroom, instead of the default group,because voice input 702 d is received within a predetermined of timefrom when the previous multicast message was sent to the sunroom. Insome embodiments, two different zones can share at least one group ofdevices (e.g., same group of devices assigned to a particular room).

As illustrated in FIG. 7X, device 700 also displays multicastnotification 708 b, which indicates that an audio multicast message hasbeen received from the group of devices assigned to sunroom 602 e. Here,device 700 receives this message at 10:19.

At FIG. 7Y, device 700 detects voice input 792 e that includes thephrase “HEY ASSISTANT, MULTICAST, LET'S GO TO DINNER” and displays userinterface 794 with indication 794 l. Indication 794 l reads: “WHAT CAN IHELP YOU WITH?” At FIG. 7Y, the time is now 10:30 (e.g., 11 minutes hasgone by since the user interface in FIG. 7X was displayed.)

At FIG. 7Z, after displaying user interface 794 in FIG. 7Y, device 700displays indication 794 m on the right side of user interface 794 anddisplays indication 794 n on the left side of user interface 794 (andbelow indication 794 m). Indication 794 m is text that reads “HEYASSISTANT, MULTICAST, LET'S GO TO DINNER,” which is the text thatcorresponds to the audio phrase included in voice input 792 e.Additionally, indication 794 n is text that reads “OK, I'VE SENT YOURMULTICAST TO 123 MAIN ST.”

As shown by table 706 in FIG. 7Z, after detecting voice input 792 e,device 700 directs (e.g., sends) an audio multicast message (e.g.,“Let's go to dinner”) to one or more devices assigned to home 602,without voice input 792 e including an identifier for home 602. Here,because more than a predetermined period of time has passed sincemulticast notification 708 b was received, device 700 intelligentlyroutes the next detected audio multicast message to the default group(group corresponding to home 602) instead of sunroom 602 e. In someembodiments, device 700 intelligently routes the detected audiomulticast message to the default group because voice input 702 e is notreceived within a predetermined of time from when the previous multicastmessage was sent to sunroom 602 e.

Further, as shown by table 706 in FIG. 7Z, the multicast message of FIG.7Z (as opposed to FIG. 7N) was not directed to bedroom speaker 636because the multicast setting for bedroom speaker 636 was turned off inresponse to detecting tap gesture 750 q in FIG. 7Q.

At some time after displaying user interface 794 in FIG. 7Z, device 700displays user interface 710 as illustrated in 7AA. At FIG. 7AA, device700 detects tap gesture 750 aa at a location that corresponds to homesettings affordance 928, which is above home location identifier 712 inFIG. 7AA.

As illustrated in FIG. 7AB, in response to detecting tap gesture 750 aa,device 700 displays user interface 797. User interface 797 includes agroup of people 797 a having one or more personal devices that areassigned to the home application (e.g., a group that corresponds to home602 a). User interface 797 also includes a multicast permissionsaffordance 797 b. At FIG. 7AB, device 700 detects tap gesture 750 ab ata location that corresponds to multicast permissions affordance 797 b.

As illustrated in FIG. 7AC, in response to detecting tap gesture 750 ab,device 700 displays user interface 799, which includes a plurality ofoptions of when to allow audio multicast messages to be sent to apersonal device. In FIG. 7AC, the options include never option 779 a, athome option 779 b, and anytime option 799 c. In FIG. 7AC, anytime option799 c is selected. Thus, in some embodiments, device 700 is configuredto receive multicast messages at any time, irrespective of the locationof device 700. In some embodiments, in response to detecting tap gesture750 c 1 at a location that corresponds to never option 779 a, device 700is configured to never receive multicast messages, irrespective of thelocation of device 700. In some embodiments, in response to detectingtap gesture 750 c 2 at a location that corresponds to at home option 799b, device 700 is configured to only receive multicast message whiledevice 700 is at a location that corresponds to home 602 or within home602 (e.g., within the threshold distance of the home or with respect towhether the device is connected to one or more networks associated withhome 602). In some embodiments, when device 700 is distance 670 anddistance 680 away from home, such as computer device 674 and Jane'sPhone 684, device 700 would not receive an audio multicast message whenat home option 799 b is selected if those distances are greater than thethreshold distance. In some embodiments, one or more options areselectable and displayed, such as an option to only receive messagewhile the device is away from the home. In some embodiments, when device700 is connected to the one or more networks associated with home 602(e.g., connected to the home Wi-Fi network), device 700 would receive anaudio multicast message when at home option 799 b is selected; and whendevice 700 is not connected to the one or more networks associated withhome 602 (e.g., not connected to the home Wi-Fi network), device 700would not receive an audio multicast message when at home option 799 bis selected (e.g., irrespective of the location of the device).

FIGS. 7AD-7AE illustrate exemplary user interfaces for displaying theuser interfaces described above on a device with a bigger screen thandevice 700, such as tablet 634. User interface 711 in FIG. 7ADcorresponds to user interface 710 discussed above. In addition, userinterface 731 in FIG. 7AE corresponds to user interface 730 discussedabove. However, any of the user interfaces discussed above, can bedisplayed and transitioned on tablet 634, using one or more similartechniques to those discussed above for FIGS. 7A-7AC.

FIGS. 7AF-7AH illustrate exemplary user interfaces for displaying theuser interfaces described above on a device with a smaller screen thandevice 700, such as smartwatch 701. User interface 713 in FIG. 7AFcorresponds to user interface 710 discussed above. Moreover, userinterface 741 in FIG. 7AG corresponds to user interface 740 discussedabove. Further, user interface 751 in FIG. 7AH corresponds to userinterface 750 discussed above. However, any of the user interfacesdiscussed above, can be displayed and transitioned on smartwatch 701,using one or more similar techniques to those discussed above for FIGS.7A-7AC.

FIGS. 8A-8B are a flow diagram illustrating a method for sending anaudio message, in accordance with some embodiments. Method 800 isperformed at a computer system (e.g., 100, 300, 500, 634, 674, 700, 701,850) in communication with a display generation component (e.g., atouch-sensitive display). Some operations in method 800 are, optionally,combined, the orders of some operations are, optionally, changed, andsome operations are, optionally, omitted.

As described below, method 800 provides an intuitive way for sending anaudio message, in accordance with some embodiments. The method reducesthe cognitive burden on a user for sending an audio message, therebycreating a more efficient human-machine interface. For battery-operatedcomputing devices, enabling a user to send audio messages faster andmore efficiently conserves power and increases the time between batterycharges.

The computer system displays (802), via the display generationcomponent, a first user interface (e.g., 730) (e.g., a user interfacefor controlling a set of external devices). The first user interfaceincludes (e.g., includes display): a first selectable user interfaceobject (e.g., 726), where selection of the first selectable userinterface object initiates a process (e.g., user interfaces 730, 740,and 750) for recording (e.g., at the computer system via a microphone)and sending a first audio message to a set of external devices (e.g.,706 in FIG. 7F) (e.g. each external device in the set of externaldevices), wherein the set of external devices includes a first set ofone or more external devices (e.g., 616) and a second set of one or moreexternal devices (e.g., 646, 648). In some embodiments, while displayingthe first selectable user interface object, the computer system receivesinput that corresponds to a selection of the first selectable userinterface object and, in response to receiving the input thatcorresponds to the selection of the first selectable user interfaceobject, the computer system displays a user interface for recording andsending an audio message to the first set of one or more devices and thesecond set of one or more devices (e.g., the set of external devices).In some embodiments, the first set of one or more devices is differentfrom the second set of one or more devices. In some embodiments, theuser interface for controlling the set of external devices includesdisplaying a plurality of representations and/or accessory control userinterface objects that corresponds to the first set of one or moredevices and the second set of one or more devices. In some embodiments,the plurality of accessory control user interface objects corresponds tothe first set of one or more devices and the second set of one or moredevices is a set of one or more selectable user interface objects. Insome embodiments, each of the plurality of accessory control userinterface objects, when selected, causes a respective accessory devicein the set of external devices to perform one or more functions and/orchange a state of the respective accessory device. In some embodiments,the set of external devices includes a type of external electronicdevice (e.g., a cellular phone, tablet computer) that is not in thefirst set of one or more devices (e.g., a smart speaker) or the secondset of one or more devices (e.g., a smart speaker). In some embodiments,set of external devices is associated with a top-level location (ordestination) (e.g., a location of a first type) (e.g., a home location,business, an office (e.g., a location that is a top-level location(e.g., a location that defines one or more locations (e.g.,sub-locations) and/or one or more regions (or zones) that are within anarea and/or perimeter that is defined by the location of the top-levellocation, a location that is at the top level of a hierarchy oflocations, a location that is linked (e.g., programmatically) to one ormore locations (e.g., sub-locations) and/or one or more regions (orzones)))). In some embodiments, set of external devices includes thefirst group of devices (e.g., a subgroup of devices) and the secondgroup of devices (e.g., a subgroup of devices). In some embodiments, asub-group of devices is associated with a sub-location (e.g., a roompositioned (e.g., in the boundary, area, or perimeter of the top-levellocation; and/or defined, identified, and/or linked (e.g.,programmatically linked) (e.g., by a user) to the top-level location) ina top-level location and/or a zone (e.g., a location that is defined bytwo or more sub-locations within (e.g., within the boundary, area, orperimeter of the top-level location and/or defined, identified, and/orlinked (e.g., by a user)) the top-level location (e.g., an entertainmentzone that includes a living room and a dining room within the top-levellocation, a personal zone that includes two or more bedrooms, livingrooms, bathrooms within the top-level location) in the top-levellocation). In some embodiments, the target group is associated with thesub-group of devices and not the top-level group of devices.

The first user interface includes a first accessory control userinterface object (e.g., one of 718) (e.g., a selectable user interfaceobject) corresponding to a first external accessory device, whereselection (e.g., 750 f 1, 750 f 2) of the first accessory control userinterface object initiates a process to transmit an instruction (e.g.,selection causes transmission of the instruction) to change a state ofthe first external accessory device (e.g., causes the door to lock orunlock, causes the light to turn on or off). In some embodiments, inresponse to receiving an input, the computer system initiates a processto transmit an instruction (e.g., selection causes transmission of theinstruction) to change a state of the first external accessory deviceand the visual appearance of the first accessory control user interfaceobject changes from a first visual appearance to a second visualappearance that is different from the first visual appearance (e.g.,pressed to depressed state (or vice-versa), a first color to a secondcolor, one or more difference characters (e.g., “on to “off” (orvice-versa))).

The computer system, while displaying the first user interface (e.g.,and the first user interface), receives (804) an input (e.g., 750 f 3)(e.g., a single user input (e.g., a tap input on a selectable userinterface object or icon that will navigate to the user interface forcontrolling the second set of one or more external device) or multipleuser inputs that are received while the first selectable user interfaceobject are displayed) that corresponds to a request to display a seconduser interface (e.g., 760) (e.g., a user interface for controlling thesecond set of one or more external devices (e.g., without controllingthe first set of one or more external devices)). In some embodiments,the computer system receives the request to display the second userinterface when the computer system receives selection of a selectableuser interface object for displaying the second user interface.

In response to receiving the request to display the second userinterface, the computer system displays (806) the second user interface(e.g., 760). The second user interface includes (e.g., includesdisplaying) a second selectable user interface object (812) (e.g., 776),where selection of the second selectable user interface object initiatesa process (e.g., user interfaces 780 a, 784, and 786 a) for recordingand sending a second audio message to the second set of one or moreexternal devices (e.g., 646, 648) without sending the second audiomessage to the first set of one or more external devices (e.g., 706 inFIG. 7M). The second user interface includes a second accessory controluser interface object (814) (e.g., one of 768) (e.g., a selectable userinterface object) corresponding to a second external accessory device(e.g., 642, 646, 648) (e.g., without corresponding to the first set ofone or more devices), where selection (e.g., 750 g 1, 750 g 2) of thesecond accessory control user interface object initiates a process totransmit an instruction (e.g., selection causes transmission of theinstruction) to change a state of the second external accessory device.In some embodiments, in response to receiving an input, the computersystem initiates a process to transmit an instruction (e.g., selectioncauses transmission of the instruction) to change a state of the secondexternal accessory device and the visual appearance of the secondaccessory control user interface object changes from a first visualappearance to a second visual appearance that is different from thefirst visual appearance (e.g., pressed to depressed state (orvice-versa), a first color to a second color, one or more differencecharacters (e.g., “on to “off” (or vice-versa))). In some embodiments,in response to receiving the request to display the second userinterface, the computer system ceases to display the first userinterface and first user interface object. In some embodiments, thesecond selectable user interface object replaces the first selectableuser interface object. In some embodiments, the second selectable userinterface object is displayed at the same location as where the firstselectable user interface object was previously displayed. In someembodiments, the second user interface replaces display of the firstuser interface. In some embodiments, the computer system cannot control(e.g., cannot cause an external device to change state) the first set ofone or more external devices from the second user interface (e.g., anaccessory control user interface object corresponds to one of the firstset of one or more devices is not displayed). In some embodiments, thefirst user interface includes displaying a set of one or morerepresentations and/or accessory control user interface objects thatcorresponds to the first set of one or more external devices (withoutdisplay a set of one or more representations that corresponds to thesecond set of one or more external devices). In some embodiments, eachof accessory control user interface objects, when selected, causes arespective device to perform one or more functions or change a state ofthe respective device. In some embodiments, while displaying the secondselectable user interface object, the computer system receives inputthat corresponds to a selection of the second selectable user interfaceobject and, in response to receiving the input that corresponds to theselection of the second selectable user interface object, the computersystem displays a user interface for recording and sending an audiomessage to the second set of one or more external devices withoutsending the audio message to the first set of one or more externaldevices. In some embodiments, while displaying the second selectableuser interface object, the computer system receives a user input (e.g.,a swipe input) that corresponds to a request to display a user interfacefor controlling the first set of one or more external devices (e.g.,without controlling the second set of one or more external devices) thatincludes displaying a selectable user interface object, that, whenselected, initiates a process for recording and sending the audiomessage to the first set of one or more external devices without sendingthe audio message to the second set of one or more external devices. Insome embodiments, an audio message includes audio that is recorded atthe computer system. In some embodiments, the process for recording andsending an audio message to an external device includes recording theaudio message using the computer system (e.g., using a microphone thatis in communication with the computer system). In some embodiments, theaudio message includes voice audio of the user of the computer system.Providing hierarchal user interfaces to control a hierarchal set ofcontrols, where each user interface corresponds to a node in therespective hierarchy, provides a user with visual feedback concerningthe sets of devices in the hierarchy of controls that can be controlsvia a respective user interface along with visual feedback concerninghow the user can intuitively navigate to another respective userinterface for controlling another respective set of devices. Providingimproved visual feedback to the user enhances the operability of thecomputer system and makes the user-sys interface more efficient (e.g.,by helping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the computer system) which,additionally, reduces power usage and improves battery life of thecomputer system by enabling the user to use the device more quickly andefficiently.

In some embodiments, the first external accessory device (e.g., 718 b)is different from the second external accessory device (e.g., 768 c). Insome embodiments, the first external accessory device is an externaldevice that is grouped under the “Home” group, but is not part of the“Dining Room” category (e.g., a light located in a bedroom) and thesecond external accessory device is an external device that is groupedunder both the “Home” group the “Dining Room” category (e.g., a lightlocated in the dining room). In contrast, in some embodiments, the firstexternal accessory device is the same device as the second externalaccessory device (e.g., they are the same light located in the diningroom, or they are the same smart speaker located in the dining room).

In some embodiments, the first selectable user interface object (e.g.,726) (e.g., while displayed as part of the first user interface) isdisplayed at a first location (e.g., location of 726 and 776) via thedisplay generation component (e.g., on the display generation component)and the second selectable user interface object (e.g. 776) (e.g., whiledisplayed as part of the second user interface) is displayed at thefirst location (e.g., location of 726 and 776) via the displaygeneration component (e.g., on the display generation component). Insome embodiments, the input received while displaying the first userinterface causes the second selectable user interface object to replacedisplay of the first selectable user interface object. Providing thefirst selectable user interface or the second selectable user interfaceobjects at the same location when prescribed conditions are met allowsthe user to quickly locate a selectable user interface object formulticasting without having to identify any different operationalcharacteristics (e.g., what particular set of devices will selectablethe particular selectable user interface object send a record audiomessage to) of the selectable user interface object. Providing improvedvisual feedback to the user enhances the operability of the computersystem and makes the user-system interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the computer system by enablingthe user to use the computer system more quickly and efficiently.

In some embodiments, the first external accessory device (e.g., 646,648) (e.g., a smart speaker, wherein playback of music can be configuredusing the first accessory control user interface object) is included inthe set of external devices (e.g., the first external accessory devicecan be configured from the first user interface and audio (recorded atthe computer system) is sent to the first external accessory deviceusing the first selectable user interface object). In some embodiments,the first user interface includes a third accessory control userinterface object (e.g., 718 b) (e.g., a selectable user interfaceobject) corresponding to a third external accessory device (e.g., 642).In some embodiments, selection of the third accessory control userinterface object initiates a process to transmit an instruction (e.g.,selection causes transmission of the instruction) to change a state ofthe third external accessory device (e.g., causes the door to lock orunlock, causes the light to turn on or off). In some embodiments, thethird external accessory device (e.g., 642) is not included in the setof external devices and selection of the first selectable user interfaceobject (e.g., 776) does not initiate a process for sending (and/orrecording) the first audio message to the third external accessorydevice (e.g., 706 in 7F). In some embodiments, the first user interfaceincludes accessory control user interface objects for some externalaccessory devices that can be controlled (e.g., turned on/off), but thatare not sent the first audio message (e.g., because the externalaccessory devices do not have a speaker for playback of the audiomessage) and the first user interface includes accessory control userinterface objects for some external accessory devices that can becontrolled (e.g., turned on/off) and that are sent the first audiomessage (e.g., because the external accessory devices have speakers forplayback of the audio message). In some embodiments, the second userinterface includes accessory control user interface objects for someexternal accessory devices that can be controlled (e.g., turned on/off),but that are not sent the second audio message (e.g., because theexternal accessory devices do not have a speaker for playback of theaudio message) and the second user interface includes accessory controluser interface objects for some external accessory devices that can becontrolled (e.g., turned on/off) and that are sent the second audiomessage (e.g., because the external accessory devices have speakers forplayback of the audio message). In some embodiments, the first set ofone or more external devices (e.g., the device in the bedroom group towhich the audio message is sent) does not include the first externalaccessory device (e.g., a light in the bedroom group to which the audiomessage is not sent) and the second set of external devices (e.g., thedevice in the dining room group to which the audio message is sent) doesnot include the second external accessory device (e.g., a light in thedining room group to which the audio message is not sent). Thus, in someembodiments, the audio message is not transmitted to all devices thatcan be controlled using the accessory control user interface objectswhich, when selected, cause a respective accessory device to perform oneor more functions and/or change a state of the respective accessorydevice. In some embodiments, an external device that can be controlled(to perform one or more functions and/or change a state of therespective accessory device) using a respective accessory control userinterface object is included in the set of external devices (e.g., thefirst or second set of external devices) and, thus, an audio message canbe transmitted to that external device and that external device can beseparately controlled.

In some embodiments, the first user interface corresponds to a top-levellocation (e.g., 602) (e.g., a structure, a home) and the second userinterface corresponds to a first sub-location (e.g., 602 d) of a firsttype (e.g., 602 a-602 e) (e.g., a room of the structure, a dining roomin the home). In some embodiments, the second set of one or moreexternal devices (e.g., 646, 648) correspond to the first sub-location(e.g., 602 d) (e.g., the second set of one or more external devices havebeen assigned to the sub-location of the first type). In someembodiments, (e.g., any of) the second set of one or more externaldevices (e.g., 646, 648) cannot be made to correspond to a secondsub-location (e.g., 602 a, 602 b, 602 c, or 602 e) (e.g., anysub-location (other than the first sub-location) of the first type(e.g., any other room of the structure) based on the second set of oneor more external devices corresponding to the first sub-location (e.g.,602 a-602 e). In some embodiments, the first user interface correspondsto a top-level location and the second user interface corresponds to asub-location (e.g., a room of the home) of the top-level location. Insome embodiments, the first room and a second room are part of a zone(e.g., a collection of rooms grouped together). In some embodiments, thesecond set of one or more external devices are devices identified (e.g.,by a user) of being located in the room to which the second userinterface corresponds.

In some embodiments, the first user interface corresponds to a top-levellocation (e.g., a structure, a home) and the second user interfacecorresponds to a third sub-location of a second type (e.g.,entertainment zone (e.g., 602 d and 602 e)) (e.g., a first zone, mainfloor) that corresponds to a plurality (e.g., 602 a-602 e) ofsub-locations of a first type (e.g., 602 a-602 e) (e.g., a plurality ofrooms). In some embodiments, the second set of one or more externaldevices corresponds to the third sub-location based on the second set ofone or more external devices being included in the plurality ofsub-locations of the first type (e.g., 602 a-602 e) (e.g., a pluralityof rooms of the structure, including a first room (e.g., a dining roomin the home) and a second room (e.g., a bedroom in the home)). In someembodiments, the first user interface corresponds to a top-levellocation and the second user interface corresponds to a sub-location(e.g., multiple rooms of the home) of the top-level location. In someembodiments, the second set of one or more external devices includes atleast one device assigned to the first room and at least one deviceassigned to the second room. In some embodiments, the top-level locationcorresponds to at least one device to which the third sub-location doesnot correspond (e.g., the home includes some devices that are notincluded in the third sub-location).

In some embodiments, the second external accessory device is associatedwith (e.g., corresponds to) the third sub-location (e.g., entertainmentzone (e.g., 602 d and 602 e)) (e.g., the first zone) of the second type.In some embodiments, the second external accessory device (e.g., 646,648) is also associated with (e.g., corresponds to) a fourthsub-location (e.g., second zone) of the second type (e.g., entertainmentzone (e.g., 602 d and 602 e)) that is different from the thirdsub-location. In some embodiments, the second zone is a sub-location(e.g., multiple rooms of the home). In some embodiments, the second zoneis associated with an external accessory device that is not associatedwith the first zone. In some embodiments, the computer system caninitiate processes for recording and sending messages to various zones.A first zone and a second zone can correspond to different sets ofdevices, some of which correspond to both the first zone and the secondzone and some of which corresponds to only one of the first zone or thesecond zone.

In some embodiments, the second set of one or more external devices(e.g., 646, 648) is associated with a first location (e.g., 602 d)(e.g., a room, a sub-location, a zone) and wherein the process forrecording and sending the second audio message to the second set of oneor more external devices without sending the second audio message to thefirst set of one or more external devices includes displaying anindication (e.g., textual indication, graphical indication) of the firstlocation (e.g., 782) (e.g., text specifying the name of the room or zonethat the audio message will be transmitted to). In some embodiments,processes for recording and sending audio messages include displaying anindication of the location that corresponds to the external devices towhich the respective audio message will be sent. In some embodiments,the first user interface corresponds to a top-level location (e.g., astructure, such as a home) and the second user interface corresponds toa sub-location (e.g., a room of the home, a zone of the home) of thetop-level location. In some embodiments, displaying the second userinterface, the computer system detects selection of (e.g., tap on, pressand hold on) the second selectable user interface object. In someembodiments, in response to detecting selection of the second selectableuser interface object, the computer system displays a recording userinterface (e.g., by replacing all or part of the second user interface)that concurrently includes: a record user interface object (e.g., aselectable record button), wherein selection of (e.g., tap on, press andhold on) the record user interface object causes recording of an audiomessage by recording audio detected by the computer system; and anindication (e.g., textual indication, graphical indication) of thesub-location (e.g., text specifying the name of the room or zone thatthe audio message will be transmitted to). In some embodiments, thesecond user interface includes the indication of the sub-location.Displaying an indication of a location that is associated with a groupof devices to which an audio message is intended to be multicast providethe user with visual feedback concerning the state of the process forrecording and sending the audio message. Providing improved visualfeedback to the user enhances the operability of the computer system andmakes the user-system interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the computer system by enablingthe user to use the computer system more quickly and efficiently.

In some embodiments, as a part of initiating a process for recording andsending an audio message to external devices, the computer systeminitiates, automatically (e.g., without intervening user input thatcorresponds to a request to initiate recording of the audio), therecording of a new audio message and displays a recording user interface(e.g., 741) that includes a dynamic indication (e.g., 744 a) of theaudio being recording.

In some embodiments, the computer system detects selection of (e.g., tapon, press and hold on) the record user interface object. In response todetecting selection of the record user interface object, the computersystem records an audio message. Subsequent to recording the audiomessage (and prior to transmitting the audio message), the computersystem provides an option to change the external devices to which themessage will be transmitted. In some embodiments, the computer systemreceives activation of a drop-down affordance and, in response, displaysone or more sub-locations (each of which correspond to a respective setof external devices). The computer system receives selection of one ofthe one or more sub-locations and a confirmation to transmit the audiomessage. In response, the computer system transmits the audio message tothe respective set of external devices corresponding to the selectedsub-location, rather than to the second set of one or more externaldevices corresponding to the second user interface.

In some embodiments, the set of external devices includes a personalmobile electronic device (e.g., 604, 634, 684) (e.g., a smart phone, acell phone, a watch). In some embodiments, the personal mobileelectronic device is not included in the first set of or more devicesand is not included in the second set of one or more external devices.In some embodiments, the personal mobile electronic device is notassigned to any sub-location of the top-level location, but is a memberof the top-level location. Thus, in some embodiments, selecting thefirst selectable user interface object initiates a process to record andsend an audio message to the set of external devices, including thepersonal mobile electronic device (and any other personal mobileelectronic devices that are members of the top-level location).

In some embodiments, the second set of one or more external devices doesnot include a personal mobile electronic device (e.g., 604, 634, 684)(e.g., a smart phone, a cell phone, a watch). In some embodiments, thepersonal mobile electronic device is not included in the first set of ormore devices and is not included in the second set of one or moredevices. In some embodiments, the set of external devices includes thepersonal mobile electronic device and, therefore, audio messages sentfrom the first user interface are transmitted to the personal mobileelectronic device (and, optionally, other mobile devices that are partof the home). In some embodiments, audio messages sent from the seconduser interface (or any user interface corresponding to a sub-location)are not transmitted to the personal mobile electronic device (and,optionally, are also not transmitted to other mobile devices that arepart of the home). In some embodiments, the computer system disallowsassigning personal mobile electronic devices to any sub-location of thetop-level location, but permits assigning personal mobile electronicdevices to the top-level location. Thus, in some embodiments, selectingthe second selectable user interface object initiates a process torecord and send an audio message to the second set of one or moredevices, which does not include the personal mobile electronic device(or any other personal mobile electronic devices that are members of thetop-level location).

In some embodiments, the computer system, subsequent to displaying thesecond user interface, receives input (e.g., 750 q) (e.g., user input;an input on a settings user interface) to disable sending audio messagesto the second set of one or more external devices (e.g., 638). In someembodiments, in response to receiving the input to disable sending audiomessages to the second set of one or more external devices, the computersystem disables (e.g., 768 a) sending of audio messages to the secondset of one or more external devices via selection of the firstselectable user interface object (e.g., 726) (e.g., without disablingsending of audio messages to the first set of one or more externaldevices via selection of the first selectable user interface object) andvia selection of the second selectable user interface object (e.g.,776). In some embodiments, after disabling sending of audio messages tothe second set of one or more external devices, navigating back to thefirst user interface and receiving selection of the first selectableuser interface object initiates a process for recording and sending anaudio message to the first set of one or more external devices withoutsending the audio message to the second set of one or more externaldevices. In some embodiments, in response to receiving the input todisable sending audio messages to the second set of one or more externaldevices, the computer system removes the second set of one or moreexternal devices from the set of external devices. In some embodiments,after disabling sending of audio messages to the second set of one ormore external devices, navigating back to the second user interface andreceiving selection of the second selectable user interface object doesnot initiate a process for recording and sending an audio message to anydevices. In some embodiments, after disabling sending of audio messagesto the second set of one or more external devices, the second userinterface does not include the second selectable user interface object(or includes a non-selectable/disabled version of the user interfaceobject) such that the process for recording and sending an audio messageto the second set of one or more external devices cannot be initiated.

In some embodiments, the first user interface corresponds to a top-levellocation (e.g., 602) (e.g., a structure, such as a home) and the seconduser interface corresponds to a sub-location (e.g., 602 a-602 d) (e.g.,a room of the home, a zone of the home) of the top-level location (e.g.,602). In some embodiments, the computer system receives (808) (e.g.,while not displaying the first user interface and not displaying thesecond user interface), at a microphone that is in communication withthe computer system (e.g., a microphone of the computer system), audioinput (e.g., 792 a-792 e) including a natural language request for adigital assistance, the audio input including: an instruction toinitiate a process to transmit an audio message (e.g., including theterm “multicast”), a location identifier (e.g., the top-level location,a sub-location), and an audio message (e.g., received subsequent to theinstruction to initiate a process to transmit an audio message and/orreceived subsequent to the identifier of the location). In someembodiments, in accordance with a determination that the locationidentifier corresponds to the top-level location (e.g., 782 a), thecomputer system transmits (810) (e.g., multicasting) the audio message,without establishing a two-way audio communication channel (e.g.,without establishing a full-duplex audio communication channel) with anydevices of the set of external devices, for playback at the set ofexternal devices, including the first set of one or more externaldevices and the second set of one or more external devices. In someembodiments, in accordance with a determination that the locationidentifier corresponds to the sub-location (e.g., 792 b, 792 c): thecomputer system transmits (812) (e.g., multicasting) the audio message,without establishing a two-way audio communication channel with anydevices of the set of external devices, for playback at the second setof one or more external devices and not for playback at the first set ofone or more external devices. In some embodiments, the computer systemtransmits the location identifier along with the audio message such thata remote device (e.g., a service, an external device of the set ofexternal devices) can distribute the audio message to the externaldevices corresponding to the respective external devices.

In some embodiments, as a part of initiating a process for recording andsending an audio message to external devices includes, the computerdisplays a recording user interface (e.g., 730, 740, 750) (e.g., byreplacing all or part of the second user interface) that concurrentlyincludes: a record user interface object (e.g., 736) (e.g., a selectablerecord button), wherein selection (e.g., 750 c) of (e.g., tap on andthen press end when audio message is complete, press and hold on whilespeaking audio message, tap on and allow the computer system to identifythe end of the audio message) the record user interface object causesrecording of an audio message by recording audio detected by thecomputer system; and a send user interface object (e.g., 756) (e.g., aselectable send button), wherein selection (e.g., 750 e) of (e.g., tapon) the second user interface object causes the computer system to sendthe audio message to the external devices. In some embodiments, therecording user interface also concurrently includes an indication (e.g.,textual indication, graphical indication) of the location (e.g.,top-level location, sub-location; text specifying the name of the roomor zone that the audio message will be transmitted to) corresponding tothe external devices to which the audio message will be sent.

In some embodiments, the computer system is in communication with one ormore input devices including a microphone. In some embodiments, as apart of the process (e.g., user interfaces 730, 740, 750) for recordingand sending the first audio message to the set of external devices, thecomputer system receives, via the microphone, a second audio input. Insome embodiments, the computer system transmits the first audio messageto the set of external devices, wherein the first audio message is basedon (e.g., includes a recording of) the second audio input.

Note that details of the processes described above with respect tomethod 800 (e.g., FIGS. 8A-8B) are also applicable in an analogousmanner to the methods described below. For example, method 800optionally includes one or more of the characteristics of the variousmethods described below with reference to methods 1000 and 1200. Forexample, the computer systems can be the same systems. For anotherexample, the audio message sent in method 800 can be the media receivedin 1000. For another example, the voice input transmitted in method 1200can be the same as the media received in method 1000. For brevity, thesedetails are not repeated below.

FIGS. 9A-9X illustrate exemplary devices and exemplary user interfacesfor displaying notifications in accordance with some embodiments. Theuser interfaces in these figures are used to illustrate the processesdescribed below, including the processes in FIG. 10.

The phrases “multicast message” and/or “audio multicast message” areused generally herein with respect to FIGS. 9A-9X to refer to multicastmessages that are sent from a device, using similar techniques asdescribed in FIGS. 7A-7AH, 8A-8B, 11A-11F, and 12. However, multicastmessage and/or audio multicast message, as used herein, are notnecessarily intended to include multicast communication that are sentusing other features of device 700.

FIG. 9A illustrates electronic device 700 displaying user interface 900.In FIG. 9A, electronic device 700 is a portable multifunction device andhas one or more components described above in relation to one or more ofdevices 100, 200, 300, and 500. With respect to the figures describedbelow, in some embodiments, device 700 is operating at a location thatis less than a threshold distance away from home 602; however, in otherembodiments, device 700 is operating at a location that is greater thana threshold distance away from home 602.

As illustrated in FIG. 9A, user interface 900 is a settings userinterface that corresponds to the home application, as discussed above(e.g., in FIG. 6 and FIGS. 7A-7AH). Here, the settings user interface isa settings page for Cam Appleseed, a person (e.g., user) who isregistered to device 700 and a person who is recognized by the homeapplication (e.g., group of people 797 a in FIG. 7AB). As illustrated inFIG. 9A, device 700 is currently configured to receive multicastnotifications, which is shown by the multicast option 900 a beingenabled in FIG. 9A. Since the multicast option 900 a is enabled, themulticast notifications will be received by device 700 as illustrated inthe FIGS. 9B-9X. In some embodiments, device 700 detects tap gesture 950a at a location that corresponds to multicast option 900 a. In someembodiments, in response to detecting tap gesture 950 a (or one or moresubsequent tap gestures), device 700 changes multicast option 900 a frombeing enabled to disabled and device 700 is not configured to receivemulticast messages that are sent based on flows described herein. Thus,in some embodiments, device 700 does not receive multicast messages whenmulticast option 900 a is disabled.

FIGS. 9B-9H illustrate multicast notifications and non-multicastnotifications being displayed over a period of time, while multicastoption 900 a is enabled. In particular, FIGS. 9B-9H illustrate thatmulticast notifications (e.g., multicast message sent using the methodsdescribed herein) that are directed to (e.g., correspond to) aparticular target group of devices decay or cease to be displayed (e.g.,after a predetermined period of time) unless a new multicastnotification that is also directed to the particular target group ofdevices is received within a with a predetermined period of time. Insome embodiments, the particular target group of devices is a group ofdevices assigned to a home (e.g., home 602 in FIG. 6), a room (e.g.,dining room 602 b, bedroom 602 c in FIG. 6), or a zone (e.g.,“entertainment zone” as described, for example, in relation to FIG. 7U).

For example, in FIG. 9B, at 10:00 AM, device 700 receives dataindicating updated content has been published by a news application.

As illustrated in FIG. 9B, in response to receiving the data, device 700displays a user interface that includes news notification 902. In FIG.9B, news notification 902 is not a multicast notification. Therefore,news notification 902 is displayed regardless of whether multicastoption 900 a is enabled or disabled. In other words, the state ofmulticast option 900 a does not impact news notification 902 because itis not a multicast notification.

At 10:02 AM, device 700 receives data indicating that a new audiomulticast message has been sent (e.g., directed to devices that includeelectronic device 700) to home 602 (“home target group”). In someembodiments, the data includes a target group identifier (e.g., “Home”)as the receiver for the new audio multicast message.

As illustrated in FIG. 9C, in response to receiving the data, device 700displays multicast notification 904, which is representative of the datareceived at 10:02 AM. In FIG. 9C, multicast notification 904 isdisplayed concurrently with news notification 902. Multicastnotification 904 includes application indication 904 a, notificationtype indication 904 b, sender indication 904 c, target indication 904 d,and time of receipt indication 904 e.

Application indication 904 a indicates an application that is providingthe notification. Here, application indication 904 a includes agraphical representation icon that is assigned to a home application andis similar to home icon 704 a in FIG. 7A.

Notification type indication 904 b indicates the notification type ofthe notification. Here, the notification type is a multicastnotification.

Sender indication 904 c indicates the sender of the audio multicastmessage. Here, the sender is a person, John, who is registered with thedevice (e.g., a device that is in the home target group) that hasinitiated the process for sending the audio multicast message. In someembodiments, when the sender is a person, an identity of the sender isestablished by one or more devices analyzing voice input correspondingto the audio multicast messages, using one or more voice identificationalgorithms. In some embodiments, when the identity of the sender cannotbe established as a person, sender indication 904 c indicates that thesender is a device (e.g., “Living Speaker”), such as the device thatsent the audio multicast message. In some embodiments, the sender is notestablished as a person, for example, a person is not established at thesender device when a person is not registered with the sender devicethat initiated the process for sending the audio multicast messageand/or multiple people are registered with the device that initiated theprocess for sending the audio multicast message.

Target indication 904 d indicates a target group of devices to which anaudio multicast message corresponds. Here, target indication 904 d isthe home target group (e.g., “From Home”). In some embodiments, thetarget group (that corresponds to the audio multicast message) is thegroup identified by the sender of the audio multicast message, asdescribed above in relation to FIGS. 7E-7F, 7J-7M, and 7S-7U. In someembodiments, the target group is identified by the sender of the audiomulticast messages based on one or more user inputs (e.g., voice input(e.g., 792 a-729 d) and/or gestures (e.g., 750 b and 750 g 3). In someembodiments, the target group is identified based on an initial audiomulticast message when a reply message is received at device 700. Forexample, device 700 can receive a reply audio multicast message with atarget group that corresponds to bedroom 602 c in certain situations,such when device 700 has previously directed an initial audio multicastmessage to the group of devices assigned to bedroom 602 c.

Time of receipt indication 904 e indicates when the notification wasdisplayed (or, in some embodiments, when data that corresponds to thenotification was received). Here, time of receipt indication 904 e showsthat the notification was displayed “NOW” or at 10:02 AM.

As illustrated in FIG. 9D, at 10:09 AM, device 700 continues to displayof news notification 902 and ceases to display multicast notification904. In some embodiments, the notification 904 is not displayed on anyuser interface of device 700 after the multicast notification ceases tobe displayed. In some embodiments, the notification 904 is not displayedon the user interface in FIG. 9D at any other time after the multicastnotification ceases to be displayed. In some embodiments, the devicereceives user input to view pending notifications (e.g., notificationsthat have not been acted on by the user), and in response, displays newsnotification 902 without displaying multicast notification 904.

As illustrated in FIG. 9D, device 700 ceases to display multicastnotification 904 (even when the display is on) because datacorresponding to a new audio multicast message corresponding to the hometarget group was not received before the decay period (e.g., 1 second,30 seconds, 60 seconds, 300 seconds, etc.) of multicast notification 904ended (and/or within a predetermined period of time). On the other hand,device 700 continues to display news notification 902 because newsnotification 902 is not a multicast notification and does not cease tobe displayed once a predetermined period of time elapses (and when thedisplay is on).

FIGS. 9E-9H illustrate examples of the technique, in addition to theexample described above in relation to FIGS. 9B-9D. In FIGS. 9B-9D, thedecay period for a multicast notification (e.g., the predeterminedperiod of time) is five minutes for illustrative purposes only. In someembodiments, the decay period is shorter or longer than five minutes.

At 10:10, device 700 receives three different sets of data. The firstset of data indicates that a new audio multicast message correspondingto the home target group has been received. The second set of dataindicates that a new audio multicast message corresponding to a group ofdevices assigned to bedroom 602 c (“bedroom target group”) has beenreceived. The third set of data indicates that a new audio multicastmessage corresponding to a group of devices assigned to theentertainment zone (“the entertainment zone target group”) (e.g., acombination of a group of devices assigned to living room 602 d andsunroom 602 e) has been received.

In some embodiments, the first set of data represents an initial audiomulticast message or a reply audio multicast message (e.g., a reply toan initial audio multicast message that device 700 sent) correspondingto the home target group. In some embodiments, the second set of dataand the third set of data represent reply audio multicast messages, thatis, audio multicast messages that are received in response to initialaudio multicast messages that were separately sent to each respectivetarget group. In some embodiments, the second set of data and the thirdset of data are not initial messages that are received from theirrespective target group because device 700 is not assigned to zone orroom target groups and cannot receive initial messages sent from thosetarget groups, as discussed above (e.g., in relation to FIGS. 7M, 7T,7V, and 7X).

As illustrated in FIG. 9E, in response to receiving the three sets ofdata, device 700 displays multicast notifications 906 a, 908 a, and 910.Device 700 also continues to display news notification 902. Asillustrated in FIG. 9E, the notifications are displayed in anotification list. The oldest notification is displayed at the bottom ofthe list (e.g., news notification 902), and the newest notification isat the top of the list (e.g., multicast notification 910). Thus, whilenotifications 906 a, 908 a, and 910 were displayed at around the sametime, notifications 906 a, 908 a, and 910 were respectively displayed inthe order shown in FIG. 9E. In some embodiments, the notifications aredisplayed in a different order or in a different arrangement.

As illustrated in FIG. 9E, notification 906 a indicates that an audiomulticast message has been sent by “John” from the home target group;notification 908 a indicates that an audio multicast message has beensent by “Jane” from the bedroom target group; and notification 910indicates that an audio multicast message has been sent by “Emma” fromthe entertainment zone target group. As illustrated in FIG. 9E, allthree of the new multicast notifications include a time receiptindication of “Now” because data was recently received (or they wererecently displayed) at 10:10 AM. However, the time receipt indicationfor news notification 902 is “10 MINS AGO” because its data was received(or the notification was displayed) at 10:00.

At 10:11 AM, device 700 receives data indicating that a new audiomulticast message corresponding to the home target group has beenreceived.

As illustrated in FIG. 9F, in response to receiving the new data, device700 displays multicast notification 906 b because data indicating a newaudio multicast message corresponding to the home target group wasreceived (e.g., data received at 10:11) before the decay period formulticast notification 906 a expired (e.g., within a predeterminedperiod of time after displaying multicast notification 906 a orreceiving the first data in FIG. 9E). Notably, device 700 does notupdate the display of the notifications 906 a and 910 and continues todisplay these notifications because data indicating a new audiomulticast message corresponding to their respective target groups hasnot been received and their decay period has not expired (e.g., theyhave not been displayed for more than five minutes without receivingdata that corresponds to a new multicast message for their targetgroups).

Multicast notification 906 b is an updated version of multicastnotification 906 a, which device 700 has ceased to display in FIG. 9F.Looking back at FIG. 9E, multicast notification 906 a indicates thatJohn sent a multicast message that corresponds to the home target group.Turning back to FIG. 9F, multicast notification 906 b also includesinformation (e.g., “John” in notification 906 b) that John sent anearlier multicast message (the multicast message originallyrepresentation by notification 906 a) that corresponds to the hometarget group. In addition, multicast notification 906 b includesinformation that one other has sent a multicast message that correspondsto the home target group. Thus, multicast notification 906 b includesinformation concerning two audio multicast messages, that is, multicastnotification 906 b includes information that is representative of thesender (“John”) of the first audio multicast message that corresponds tothe home target group and information that is representative of thesender (e.g., “one other”) of the second audio multicast message thatcorresponds to the home target group.

As illustrated in FIG. 9F, multicast notification 906 b is displayed atthe top of the list with a time receipt indication of now because it isthe most recent notification. Notably, multicast notification 906 b'stime receipt indication changes in FIG. 9F to reflect the time at whichthe new audio multicast message corresponding to the home target groupwas received (e.g., data received at 10:11), although multicastnotification 906 a first displayed (or data that corresponds tomulticast notification 906 a was first received) at 10:10 AM in FIG. 9E.In some embodiments, multicast notification 906 a is displayedconcurrently with multicast notification 906 b. In some embodiments,when multicast notification 906 a is displayed concurrently withmulticast notification 906 b, the time receipt indication of multicastnotification 906 a is updated to read, “NOW.” In some embodiments,updating the time indication of multicast notification 906 a extends thedecay period of multicast notification 906 a. In some embodiments,receiving the data at FIG. 9F extends the decay period of multicastnotification 906 a (e.g., or the content of multicast notification 906 abecause multicast notification 906 b includes content representative ofthe audio multicast message represented by multicast notification 906a).

As illustrated in FIG. 9F, notifications 910, 908 a, and 902, which arelisted respectively from most recent to least recent, are belowmulticast notification 906 b. In addition, the time receipt indicationsof notifications 910, 908 a, and 902 have been updated in FIG. 9F toreflect the length of time that each has respectively been displayed (orthe time that has passed since data has been received that correspondsto each respective notification).

At 10:12 AM, device 700 receives data indicating that a new audiomulticast message corresponding to the bedroom target group has beenreceived.

As illustrated in FIG. 9G, in response to receiving the data, device 700displays multicast notification 908 b because data indicating a newaudio multicast message corresponding to the bedroom target group wasreceived (e.g., data received at 10:12) before the decay period formulticast notification 908 b expired (e.g., within a predeterminedperiod of time after displaying multicast notification 906 b orreceiving the second data in FIG. 9E). Notably, device 700 does notupdate notifications 908 b and 910 and continues to display thenotifications 908 b and 910 because data indicating a new audiomulticast message corresponding to their respective target groups hasnot been received, and their decay period has not expired. In someembodiments, multicast notification 908 b is displayed using similartechniques to those described above in FIG. 9F with respect to multicastnotification 906 b.

Multicast notification 908 b is an updated version of multicastnotification 908 a, which device 700 has ceased to display in FIG. 9G.Looking back at FIG. 9E, multicast notification 908 a indicates thatJane sent a multicast message that corresponds to the bedroom targetgroup. Turning back to FIG. 9G, multicast notification 908 b indicatesthat two multicast messages have been sent by Jane that correspond tothe bedroom target group. Here, instead of multicast notification 908 bincluding a new sender, multicast notification 908 b displays contentthat is representative of an increased amount of notifications that weresent by the respective sender when compared to multicast notification908 a. In some embodiments, multicast notification 908 a includes arepresentation of the amount of notifications that were received fromthe bedroom target group, irrespective of whether a different sender hassent the new multicast message.

As illustrated in FIG. 9H, multicast notification 908 b is displayed atthe top of the list, and notifications 906 b, 910, and 902 are displayedunder multicast notification 908 b in order from most recent to leastrecent.

As illustrated in FIG. 9H, at 10:15 AM, device 700 ceases to displaymulticast notification 910 because the decay period for multicastnotification 910 has expired. In other words, device 700 has notreceived data indicating a new audio multicast message corresponding tothe target group of multicast notification 910 (e.g., “the entertainmentzone”) within a predetermined amount of time (or before the end of thedecay period of notification 910) of when multicast notification 910 wasinitially displayed (or data indicating an audio multicast messagecorresponding to the entertainment target group was received). As such,with reference to FIG. 9E, device 700 continues to display contentrepresentative of the first set of data (e.g., John in multicastnotification 906 b) and content representative of the second set of data(e.g., one of the multicasts in multicast notification 908 b) and nolonger displays content representative of the third set of data (e.g.,multicast notification 910) because data corresponding to the respectivetarget groups indicated in the first and second sets of data wasreceived before each respective notification's decay period ended whiledata corresponding to the target group indicated in the third set ofdata was not received before the decay period of multicast notification910 ended.

FIGS. 9I-9P illustrate user interfaces for playback of received audiomulticast messages. In particular, FIGS. 9I-9K illustrate a scenariowhere multiple audio multicast messages are played back automatically inresponse to a single input. In addition, FIG. 9L-9P illustrate ascenario where one received audio multicast message is selected from agroup of other received audio multicast messages and played back. AtFIG. 9H, device 700 detects tap gesture 950 h at a location thatcorresponds to multicast notification 906 b.

As illustrated in FIG. 9I, in response to detecting tap gesture 950 h,device 700 displays user interface 920. User interface 920 is a userinterface for displaying the audio multicast messages that are indicatedin multicast notification 906 b, which are audio multicast messages thatcorrespond to the home target group (as shown by “MULTICAST FROM HOME”in FIG. 9I). User interface 920 includes graphical sender indication 924a and graphical sender indication 924 b, where each represents one ofthe audio multicast messages represented in multicast notification 906b. Here, graphical sender indication 924 a is larger than graphicalsender indication 924 b and in the center of the user interface, whichindicates that the audio multicast message that corresponds to graphicalsender indication 924 a is currently selected for playback. Graphicalsender indication 924 a is an image of a speaker device andrepresentative of data corresponding to the audio multicast message sentby “one other” that was received at 10:11 AM in FIG. 9F. And graphicalsender indication 924 b is an image of John (i.e., 797 a including animage of John A. in FIG. 7AB) and representative of data corresponds tothe audio multicast message sent by John that was received at 10:10 AMin FIG. 9E. In some embodiments, the audio messages are displayed in theorder that they were received (e.g., graphical sender indication 924 band graphical sender indication 924 b are switched in FIG. 9E). Userinterface 920 further includes current sender indication 922 a, whichindicates a name that is assigned to the speaker or device that isdisplayed in the middle of the user interface (or the largest graphicalsender indication). Here, current sender indication 922 a indicates thatthe speaker has the name of “Living Room,” and an audio multicastmessage from the sender is currently playing or can currently be playedback by a selection of playback affordance 926. In addition, device 700also displays remaining indication 928, which shows that the currentlyselected audio multicast message has a length of ten seconds that can beplayed back, and count indication 930, which represents that theselected audio multicast message is message 1 of 2 of the total numberof audio multicast messages that correspond to the home target group. Insome embodiments, when a setting is enabled to allow transcription ofthe audio multicast message, user interface 920 includes a textualrepresentation of the currently selected audio multicast message.

As illustrated in FIG. 9I, device 700 receives data indicating that anew audio multicast message that corresponds to the bedroom target grouphas been received and displays multicast notification 940 in response toreceiving the data. In some embodiments, device 700 updates the list ofnotifications based on this received data, as discussed in relation toFIG. 9L below. At FIG. 9I, device 700 receives tap gesture 950 i at alocation that corresponds to playback affordance 926.

As illustrated in FIGS. 9J-9K, in response to detecting tap gesture 950i, device 700 initiates playback of media. As illustrated in FIG. 9J,device 700 plays back the last received audio multicast message thatcorresponds to the home target group (e.g., received at 10:11 in FIG. 9Fabove). In FIG. 9J, while playing back the currently selected audiomulticast message (e.g., audio multicast message from “one other”received at 10:11), device 700 displays time indication 932 and updatesindications 928 and 930. As illustrated in FIG. 9K, device 700 iscurrently playing back the first audio multicast message thatcorresponds to the home target group (e.g., received at 10:10 from Johnin FIG. 9E above). Notably, device 700 plays back the first audiomulticast message, automatically (e.g., without receiving additionalinput after detecting tap gesture 950 i) after completing playback ofthe last audio multicast message that corresponded to the home targetgroup.

As illustrated in FIG. 9L, sometime after displaying the user interfacein FIG. 9K, device 700 re-displays the list of notifications. Here,notification 902 b has ceased to be displayed. In some embodiments,notification 902 b has ceased to be displayed because a gesture wasdetected at a location corresponding to the notification and/or one ormore of the audio multicast messages were played back in FIGS. 9J-9K. Insome embodiments, notification 902 b ceases to be displayed because itsdecay period has ended (e.g., at 10:16 AM) before a new multicastcorresponding to the home target group has been received.

In FIG. 9L, device 700 is also displaying multicast notification 908 cand has ceased to display multicast notification 908 b. Multicastnotification 908 c is an updated version of multicast notification 908b, which has been updated to reference data corresponding to the audiomulticast message that was received in FIG. 9I, which was represented bymulticast notification 940 in FIG. 9I. Thus, while multicastnotification 908 b included content indicating that two multicastnotifications have been received that correspond to the bedroom targetgroup, multicast notification 908 c has been updated to include contentindicating that three multicast notifications have been received thatcorrespond to the bedroom target group. At FIG. 9L, device 700 detectspress-and-hold gesture 950 l at a location that corresponds to multicastnotification 908 c. In some embodiments, device 700 detectspress-and-hold gesture 950 l while the gesture continues to be detected(or held). In some embodiments, device 700 detects press-and-holdgesture after the gesture has been released.

As illustrated in FIG. 9M, in response to detecting press-and-holdgesture 950 l, device 700 displays user interface 960 (e.g., after anend of press-and-hold gesture 950 l, while continuing to detectingpress-and-hold gesture 950 l). User interface 960 includes similar userinterface elements to user interface 920 (as discussed above in relationto FIGS. 9I-9K). However, user interface 960 has less area than userinterface 920. While user interface 960 is a user interface for playbackaudio multicast messages from the bedroom target group, device 700 canuse one or more techniques as described above in relation to userinterface 920. In some embodiments, in response to detecting user input950 m 1 on open affordance 962, device 700 ceases to display userinterface 960 and displays a user interface that is similar in size touser interface 920. At FIG. 9M, device 700 detects swipe gesture 950 m 2at a location that corresponds to user interface 960.

As illustrated in FIG. 9N, in response to detecting a first gesture(e.g., swipe gesture 950 m 2), device 700 changes the currently selectedaudio multicast message from the last received audio multicast messagecorresponding to the bedroom target group (e.g., where its correspondingdata was received in FIG. 9I) to the second-to-last received audiomulticast message corresponding to the bedroom target group (e.g., whereits corresponding data was received in FIG. 9G). At FIG. 9N, device 700detects swipe gesture 950 n at a location that corresponds to userinterface 960.

As illustrated in FIG. 9O, in response to detecting a second gesture(e.g., swipe gesture 950 n), device 700 changes the currently selectedaudio multicast message from the second-to-last received audio multicastmessage corresponding to the bedroom target group (e.g., where itscorresponding data was received in FIG. 9F) to the first received audiomulticast message corresponding to the bedroom target group (e.g., whereits corresponding data was received in FIG. 9E). At FIG. 9O, device 700detects tap gesture 950 o at a location that corresponds to userinterface 960 (e.g., on playback affordance 926).

As illustrated in FIG. 9P, in response to detecting tap gesture 950 o,device 700 initiates playback of the first received audio multicastmessage (e.g., currently selected) corresponding to the bedroom targetgroup without initiating playback of the other audio multicast messages.In some embodiments, at FIG. 9P (or 9I-9J), device 700 detects tapgesture 950 p at a location that corresponds to multicast affordance736; and, in response to detecting tap gesture 950 p at a location thatcorresponds to multicast affordance 736, device 700 starts recordingmedia, via one or more input devices, as described above (e.g., in FIGS.7C and 7I). In some embodiments, at FIG. 9P, device 700 detects one ormore voice inputs, such as voice input 720 a (as described above), tosend an audio multicast message corresponding to the bedroom targetgroup in order to replay to Jane.

FIGS. 9Q-9V illustrate exemplary user interfaces for displaying the userinterfaces described above on a device with a smaller display thandevice 700, such as smartwatch 701. Any of the user interfaces discussedabove can be displayed and transitioned on smartwatch 701 (e.g., userinterfaces 970, 972, 974, 976, 978, and 980) (or tablet 634), using oneor more similar techniques to those discussed above for FIGS. 9A-9P.

FIGS. 9W-9X illustrate exemplary user interfaces for displaying the userinterfaces described above on a device, such as computer device 674. Asdescribed above, computer device 674 is in communication with one ormore devices that sends instructions for computer device 674 to displayone or more user interfaces, in some embodiments. Any of the userinterfaces discussed above can be displayed and transitioned on computerdevice 674 (e.g., user interfaces 996 and 998), using one or moresimilar techniques to those discussed above for FIGS. 9A-9P.

FIG. 10 is a flow diagram illustrating a method for displayingnotifications using a computer system in accordance with someembodiments. Method 1000 is performed at a computer system (e.g., 100,300, 500, 634, 674, 700, 701, 850) in communication with a displaygeneration component. Some operations in method 1000 are, optionally,combined, the orders of some operations are, optionally, changed, andsome operations are, optionally, omitted.

As described below, method 1000 provides an intuitive way for displayingnotifications. The method reduces the cognitive burden on a user forview notifications, thereby creating a more efficient human-machineinterface. For battery-operated computing devices, enabling a user toaccess notifications faster and more efficiently conserves power andincreases the time between battery charges.

The computer system receives (1002) first data corresponding to firstmedia.

The computer system displays (1004) (e.g., in response to receiving thefirst data), via the display generation component, until an end of afirst time period (e.g., a predetermined first time period), a firstnotification (e.g., 906 a, 908 a, 910) with an indication of the firstmedia (e.g., 904 b, 904 c) (e.g., a notification corresponding to thefirst data) (e.g., a notification selectable to initiate a process toplay the first media).

The computer system, after receiving the first data (e.g., after thestart of display of the first notification), receives (1006) second datacorresponding to second media, wherein the second media (e.g., 932 in9J) is different from the first media (e.g., 932 in 9K).

The computer system, in accordance with a determination that the secondmedia meets a set of one or more criteria (and, in some embodiments, theone or more criteria includes a criterion that is met when the secondmedia is directed to the same target group as the first media) and inaccordance with a determination based on (1) a time that the second datawas received and (2) a time that the first data was received (e.g., adetermination that the second data was received within a predefinedamount of time after the first data was received), displays (1008)(e.g., in response to receiving the second data), via the displaygeneration component, until an end of a second time period. In someembodiments, the second time period is based on receipt of the seconddata. In some embodiments, the second time period is not based onreceipt of the first data. In some embodiments, the second time periodis a predetermined second time period), a second notification (e.g., 906b, 908 b, 908 c) with an indication (e.g., indication in 906 b (e.g.,“John”), 908 b (e.g., “2”)) of the first media (e.g., 932 in 9K) (e.g.,a notification corresponding to at least the first data) (e.g., anotification selectable to initiate a process to play at least the firstmedia) (e.g., a notification indicating receipt of at least the firstdata), where an end of the first time period is different from an end ofthe second time period. In some embodiments, the second notification isthe first notification. In some embodiments, the second notification isdifferent from the first notification (e.g., the second notification isa notification combining (1) information corresponding to the firstmedia and (2) information corresponding to the second media). In someembodiments, the second notification is displayed (e.g., initiallydisplayed) after the first time period. In some embodiments, whenreceipt of the second media is within a predetermined time of receipt ofthe first media, a length of time is extended for which informationcorresponding to the first media is displayed. In some embodiments, thesecond time period is determined based on a difference between the whenthe first data was received and when the second data was received. Insome embodiments, the media (e.g., first media, second media) is anaudio message that was recorded at another device and transmitted to agroup of devices, including the computer system receiving the first dataand the second data. In some embodiments, the set of one or morecriteria includes a criterion that is met when the second media isreceived prior to an end of a display period (e.g., a duration duringwhich the notification is valid, a duration during which thenotification will be displayed in a collection of notifications) of thefirst media. In some embodiments, the set of one or more criteriaincludes a criterion that is met when the second media is received priorceasing to display the first notification (e.g., via activation of thefirst notification, via receiving user input clearing the firstnotification). Dynamically displaying a second notification thatincludes an indication that is included in a first notification that wasrelated to the second notification when certain criteria are metprovides the user with feedback about the relationship between thesecond notification and the first notification while also allowing theindication that is included in the first notification to continue to bedisplayed even if the first notification has ceased to displayed.Providing improved visual feedback to the user enhances the operabilityof the computer system and makes the user-system interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the computersystem) which, additionally, reduces power usage and improves batterylife of the computer system by enabling the user to use the computersystem more quickly and efficiently.

In some embodiments, the first notification (e.g., 906 a) is beingdisplayed at the time the second data is received (e.g., in accordancewith the determination that the second media meets the set of one ormore criteria). In some embodiments, as a part of displaying the secondnotification (e.g., 906 b), the computer system ceases to display thefirst notification (e.g., in FIG. 9E). In some embodiments, inaccordance with the determination that the second media meets the set ofone or more criteria (and, in some embodiments, the one or more criteriaincludes a criterion that is met when the second media is directed tothe same target group as the first media) and in accordance with adetermination that the second data was received within a predefinedamount of time after the first data was received, ceasing to display thefirst notification. Ceasing to display the first notification when asecond notification includes information concerning the firstnotification provides the user with feedback about the information inthe first notification without cluttering the user interface. Providingimproved visual feedback to the user enhances the operability of thedevice and makes the user-system interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the computer system) which,additionally, reduces power usage and improves battery life of thecomputer system by enabling the user to use the computer system morequickly and efficiently.

In some embodiments, the first media is a first audio message (e.g., 932in 9K) (e.g., an audio message recorded (using a microphone) at anexternal device and transmitted by the external device to a group ofdevices, an audio message that was directed to (e.g., transmitted to) aplurality of external devices that includes the computer system). Insome embodiments, the second audio message is a second audio messagethat was directed (or transmitted) to a second plurality of externaldevices (e.g., same as the plurality of external devices, different fromthe plurality of external devices). Displaying a notification that isassociated with audio messages that have been multicast provides theuser with visual feedback concerning the type of notification, whichcause the user to identify that the notification will decay and needs tobe responded to in a predetermined period of time. Providing improvedvisual feedback to the user enhances the operability of the computersystem and makes the user-system interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the computer system) which,additionally, reduces power usage and improves battery life of thecomputer system by enabling the user to use the computer system morequickly and efficiently.

In some embodiments, the first notification (e.g., 904) includes anindication (e.g., 904 d) of a group of a plurality of external devicesto which the first media (e.g., 932 in 9K) was directed (e.g., to whichthe first data corresponding to the media was transmitted). In someembodiments, the first data identifies the group of the plurality ofexternal devices to which the first media was directed. In someembodiments, the plurality of external devices includes the computersystem. Displaying a notification that includes an indication of thegroup of devices in which the first media is directed provides the userwith feedback about the devices that sent and/or will receive a replaymessage. Providing improved visual feedback to the user enhances theoperability of the computer system and makes the user-system interfacemore efficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the computersystem) which, additionally, reduces power usage and improves batterylife of the computer system by enabling the user to use the computersystem more quickly and efficiently.

In some embodiments, in response to receiving the first data, thecomputer system adds the first notification (e.g., 906 a) to acollection of notifications (e.g., notifications displayed in FIGS.9B-9H) (e.g., a collection of notifications which are accessible fordisplay, which are displayed in response to user input requesting todisplay the collection of notifications, a listing of recentnotifications) until an end of the first time period (e.g., apredetermined first time period) In some embodiments, in accordance withreaching the end of the first time period, the computer system removesthe first notification (e.g., 904) from the collection of notifications.In some embodiments, in accordance with receiving, prior to the end ofthe first time period, the second data that corresponds to the secondmedia and in accordance with a determination that the second media meetsthe set of one or more criteria, the computer system removes the firstnotification (e.g., 906 a) from the collection of notifications; andadds the second notification (e.g., 906 b) to the collection ofnotifications. In some embodiments, in accordance with receiving, priorto the end of the first predetermined time period, data that correspondsto media and in accordance with a determination that the second mediadoes not meet the set of one or more criteria: adding the secondnotification to the collection of notifications without removing thefirst notification from the collection of notifications. In someembodiments, the computer system removes the first notification from thecollection of notifications in accordance with the end-of-displaycondition being met for the first notification. Choosing to remove anotification when no additional data (e.g., no updates or no new media)that is related to the notification has been received within a certainperiod of time allows the device to avoid providing stale notifications(e.g., old, non-relevant) to the user that would clutter the UI, withoutrequiring the user to manually delete the notifications. Performing anoptimized operation when a set of conditions has been met withoutrequiring further user input enhances the operability of the computersystem and makes the user-system interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the computer system) which,additionally, reduces power usage and improves battery life of thecomputer system by enabling the user to use the computer system morequickly and efficiently.

In some embodiments, the first data identifies a target group of aplurality of external devices to which the first media was directed(e.g., transmitted to). In some embodiments, while displaying anotification (e.g., 906 a, 908 a, 910) (e.g., the first notification)corresponding to the first media (e.g., and which, when the notificationis activated, initiates a process for playing the first media (e.g.,first audio message)), the computer system receives (1010) third datacorresponding to third media. In some embodiments, in response to (1012)receiving the third data corresponding to the third media and inaccordance with a determination that the third media meets the set ofone or more criteria (wherein, in some embodiments, the one or morecriteria includes a criterion that is met when the media is directed tothe same target group of devices as the first media), the computersystem extends (1014) the duration during which the notification (e.g.,the first notification, the second notification) corresponding to thefirst media (e.g., and which, when the notification is activated,initiates a process for playing the first media (e.g., first audiomessage)) is displayed (e.g., 906 b in FIG. 9F). In some embodiments, inresponse to (1012) receiving the third data corresponding to the thirdmedia and in accordance with a determination that the third media doesnot meet the set of one or more criteria (e.g., the third media isdirected to a different set of target devices as compared to the firstmedia), the computer system forgoes (1016) extending the duration duringwhich the notification corresponding to the first media is displayed(e.g., 908 a, 910 in FIG. 9F). In some embodiments, as the computersystem receives additional data that corresponds to media that meet theset of one or more criteria, the device continues to extend the durationduring which notifications are displayed via the display generationcomponent as long as the immediately previous notification that wasdisplayed in response to the media meeting the set of one or morecriteria has not yet ceased to be displayed based on reaching the end ofa time period. Thus, as audio messages are received directed to the sameset of target devices (e.g., external devices), the notification forthat chain of audio messages continues to be displayed as long as themost recently received audio message does not reach the correspondingpredetermined time period for display. Choosing to whether to extend adecay period for a notification based on whether updated informationrelated to the notification has been received for a predetermined periodof time allows the device to providing fresh (e.g., new, relevant)notifications to the user while avoiding the provision of stalenotifications (e.g., old, non-relevant) to the user that would clutterthe UI, without requiring the user to manually update/delete thenotifications. Performing an optimized operation when a set ofconditions has been met without requiring further user input enhancesthe operability of the computer system and makes the user-systeminterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thecomputer system) which, additionally, reduces power usage and improvesbattery life of the computer system by enabling the user to use thecomputer system more quickly and efficiently.

In some embodiments, in accordance with the determination that thesecond media meets the set of one or more criteria, the secondnotification (e.g., that meets the set of one or more criteria) includesan indicator (e.g., a number) that is based on (e.g., that indicates) anumber (e.g., in 908 a and 908 b) of media, including the first media,(e.g., the number of audio messages) represented by the secondnotification. In some embodiments, each subsequent notification afterthe first notification in a chain of notifications includes anindication of the number of media (e.g., number of audio messages)represented by the notification being displayed. For example, if anotification is the 5th notification in the chain, the notificationindicates that the notification corresponds to 5 received media (e.g., 5audio messages). In some embodiments, in accordance with a determinationthat the second media does not meet the set of one or more criteria, thesecond notification (e.g., that meets the set of one or more criteria)does not include an indicator (e.g., a number) that is based on (e.g.,that indicates) a number of media, including the first media, (e.g., thenumber of audio messages) represented by the second notification.Updating an indication of the number of media items (audio multicastmessage (e.g., an audio message that is multicast to one or moredevices)) that has been received from a target group of devices providesthe user with feedback about the current state the number of media itemsthat have been received for the respected target group of devices.Providing improved visual feedback to the user enhances the operabilityof the computer system and makes the user-system interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the computersystem) which, additionally, reduces power usage and improves batterylife of the computer system by enabling the user to use the computersystem more quickly and efficiently.

In some embodiments, the second notification includes the indication ofthe first media. In some embodiments, while displaying the secondnotification (e.g., 602 b), the computer system receives input (e.g.,950 h) directed to the second notification. In some embodiments, inresponse to receiving the input directed to the second notification(e.g., and in accordance with a determination that the second mediameets the set of one or more criteria), the computer system initiates aprocess (e.g., 920, 922 in FIGS. 9I-9J) for playing back the first mediaand the second media (e.g., displaying a user interface for playing backthe first media and the second media, starting playback of the firstmedia and, subsequently, the second media). In some embodiments,initiating a process for playing back the first and the second mediaincludes automatically (e.g., in response to receiving the inputdirected to the second notification without intervening user input)playing back the first media and, subsequently, the second media anddisplaying a user interface (e.g., 741) for playing back the firstmedia. In some embodiments, in response to receiving the input directedto the second notification and in accordance with a determination thatthe second media does not meet the set of one or more criteria, as apart of initiating a process for playing back the second media withoutplaying back the first media (e.g., displaying a user interface forplaying back the second media and not the first media), the computersystem starts playback of the second media and, not automaticallysubsequently playing back the first media. In some embodiments, inaccordance with a determination that the second media does not meet theset of one or more criteria and while displaying the first notification,the computer system receives input directed to the first notification,and in response to receiving input directed to the first notification,initiates a process for playing back the first media without playingback the second media.

In some embodiments, the second media meets the set of one or morecriteria (e.g., the second media is directed to the same externaldevices as the first media). In some embodiments, while displaying thesecond notification (e.g., 908 c), the computer system receives input(e.g., 950 l) directed to the second notification. In some embodiments,in response to receiving the input directed to the second notification(e.g., and in accordance with a determination that the second mediameets the set of one or more criteria), the computer system displays auser interface (e.g., 960) for playing back the first media (e.g., aspart of initiating a process for playing back the first media and thesecond media), wherein the user interface for playing back the firstmedia includes a record user interface object (e.g., a selectable recordbutton). In some embodiments, selection (e.g., 950 p) of the record(e.g., 736) user interface object causes, in accordance with adetermination that the first data identifies a first group (e.g.,bedroom devices) of a plurality of external devices to which the firstmedia was directed (e.g., transmitted), the computer system to initiatea process (e.g., 730, 740, 750) for transmitting an audio message to thefirst group of plurality of external devices (e.g., and not the secondgroup). In some embodiments, selection (e.g., 950 p) of the record(e.g., 736) user interface object causes, in accordance with adetermination that the first data identifies a second group (e.g., homedevices) of a plurality of external devices to which the first media wasdirected (e.g., transmitted), different from the first group, thecomputer system to initiate a process (e.g., 780 a, 784, 786 a) fortransmitting an audio message to the second group of plurality ofexternal devices (e.g., and not the first group). In some embodiments,the computer system detects selection of (e.g., tap on, press and holdon) the record user interface object, and in response to detectingselection of the record user interface object initiates thecorresponding operation, as noted above. In some embodiments, the set ofexternal devices is associated with a top-level location (ordestination) (e.g., a location of a first type) (e.g., a home location,business, an office (e.g., a location that is a top-level location(e.g., a location that defines one or more locations (e.g.,sub-locations) and/or one or more regions (or zones) that are within anarea and/or perimeter that is defined by the location of the top-levellocation, a location that is at the top level of a hierarchy oflocations, a location that is linked (e.g., programmatically) to one ormore locations (e.g., sub-locations) and/or one or more regions (orzones)))). In some embodiments, initiating a process for transmitting anaudio message includes recording an audio message by recording audiodetected by the computer system. In some embodiments, the user interfacefor playing back the first media includes an indication (e.g., textualindication, graphical indication) of the group (e.g., a top-levellocation, sub-location) (e.g., text specifying the name of the room orzone that the audio message will be transmitted to).

In some embodiments, while displaying a user interface (e.g., 960) forplaying back the first media (e.g., as part of initiating a process forplaying back the first media and the second media), the computer systemreceives user input (e.g., 950 i, 950 m 2, 950 n) (e.g., via one or moreinput devices, using a touch-sensitive surface). In some embodiments, inresponse to receiving the user input and in accordance with adetermination that the user input (e.g., 950 i) is directed to selectionof a play user interface object (e.g., a tap on a play button displayedas part of the user interface for playing back the first media), thecomputer system plays back the first media (and, optionally,automatically subsequently playing back the second media and switchingto the user interface for playing back the second media). In someembodiments, in response to receiving the user input and in accordancewith a determination that the user input is a swipe gesture (e.g., 950 m2, 950 n) directed to the user interface for playing back the firstmedia (e.g., a left swipe gesture on the user interface for playing backthe first media), the computer system replaces display of the userinterface for playing back the first media with display of a userinterface for playing back the second media (e.g., which includes a playbutton for playing the second media without playing the first media). Insome embodiments, in response to receiving the user input and inaccordance with a determination that the user input (e.g., 950 i) isdirected to selection of a play user interface object (e.g., a tap on aplay button displayed as part of the user interface for playing back thefirst media), the computer system plays back the second media (e.g., themost recently received media corresponding to the notification) andsubsequently plays back the first media.

In some embodiments, while displaying a user interface (e.g., 720 ofFIGS. 15B-15E) for playing back the first media (e.g., as part ofinitiating a process for playing back the first media and the secondmedia) and while playing back the first media, the computer systemreceives input. In some embodiments, in response to receiving the userinput and in accordance with a determination that the user input isdirected to a first location (e.g., location of 736 on user interface720) on the user interface for playing back the first media, computersystem displays a user interface (e.g., 741) for sending and recordingnew media (e.g., one or more audio multicast messages) (and, optionally,automatically starts to record audio for sending). In some embodiments,in response to receiving the user input and in accordance with adetermination that the user input is directed to a second location(e.g., a location of 1534) on the user interface for playing back thefirst media, the computer system replaces display of the user interfacefor playing back the first media (e.g., that corresponds to 1524 a) withdisplay of a user interface for playing back the second media (e.g.,that corresponds to 1524 b), wherein the second location on the userinterface for playing back the first media is different from the firstlocation on the user interface for playback the first media. Inaccordance with a determination that the user input is directed to thefirst location and is a tap input, the computer system automaticallyrecords audio (e.g., via one or more microphones) until a subsequentinput is detected (e.g., a subsequent input that is detected in the samelocation (or near the first location) as the first location). Inaccordance with a determination that the user input is directed to thefirst location and is a press-and-hold gesture, the computer systemautomatically records audio (e.g., via one or more microphones) untilthe end of the user input is detected. In some embodiments, whiledisplaying the second notification, the computer system receives input(e.g., 950 h) directed to the second notification. In some embodiments,in response to receiving the input directed to the second notification,the computer system ceases to display the second notification. In someembodiments, once the second notification is activated, the secondnotification is removed from a collection of notifications (e.g., thatare user-accessible for display, that are displayed in response to userinput requesting to display the collection of notifications, a listingof recent notifications received at the computer system). In someembodiments, the collection of notifications continues to include othernotifications (e.g., email notifications, instant messagingnotifications). Choosing to remove a notification when the notificationhas been interacted with allows the device to avoid providing stalenotifications (e.g., old, non-relevant) to the user that would clutterthe UI, without requiring the user to manually delete the notifications.Performing an optimized operation when a set of conditions has been metwithout requiring further user input enhances the operability of thecomputer system and makes the user-system interface more efficient(e.g., by helping the user to provide proper inputs and reducing usermistakes when operating/interacting with the computer system) which,additionally, reduces power usage and improves battery life of thecomputer system by enabling the user to use the computer system morequickly and efficiently.

In some embodiments, the computer system displays a user interface forconfiguring whether to include the computer system in a group ofplurality of external devices that receive data corresponding to media(e.g., device that receive audio messages directed to external devicesof the home) directed to a location (e.g., a top-level location (ahome), a sub-location (a room of the home), wherein the user interfacefor configuring whether to include the computer system in a groupincludes: a first option (e.g., 799 c) to (e.g., always) include thecomputer system in the group of plurality of external devices thatreceive data corresponding to media (e.g., device that receive audiomessages directed to external devices of the home) directed to thelocation, regardless of a current physical location of the computersystem (e.g., as determined by a location sensor of the computersystem); and a second option (e.g., 799 a) to (e.g., never) not includethe computer system in the group of plurality of external devices thatreceive data corresponding to media (e.g., device that receive audiomessages directed to external devices of the home) directed to thelocation, regardless of the current physical location of the computersystem (e.g., as determined by a location sensor of the computersystem). In some embodiments, the user can configure the computer systemto register/unregister to receive data corresponding to the media or toenabled/disable display of notifications based on received datacorresponding to the media.

In some embodiments, the computer system displays a user interface forconfiguring whether to include the computer system in a group ofplurality of external devices that receive data corresponding to media(e.g., device that receive audio messages directed to external devicesof the home) directed to a location (e.g., a top-level location (ahome), a sub-location (a room of the home), wherein the user interfacefor configuring whether to include the computer system in a groupincludes a third option (e.g., 77 b) to include the computer system(e.g., 700) in the group of plurality of external devices that receivedata corresponding to media (e.g., device that receive audio messagesdirected to external devices of the home) directed to the location whenthe computer system (e.g., 700) is currently at a physical location(e.g., within a geo-fenced perimeter of the home, within a particularroom of the home, as determined by a location sensor of the computersystem) corresponding to the location and not include the computersystem (e.g., 700) in the group of plurality of external devices thatreceive data corresponding to media (e.g., device that receive audiomessages directed to external devices of the home) directed to thelocation when the computer system (e.g., 700) is not currently at thephysical location (e.g., within a geo-fenced perimeter of the home,within a particular room of the home, as determined by a location sensorof the computer system) corresponding to the location. In someembodiments, the user can provide input to configure the computer systemto only receive data corresponding to media (and/or to only displaynotifications) directed to a group of devices of a location when thecomputer system is at that location (e.g., only get messages directed tothe home when the computer system is in the home, only get messagesdirected to a room when the computer system is in that room).

Note that details of the processes described above with respect tomethod 1000 (e.g., FIG. 10) are also applicable in an analogous mannerto the methods described below/above. For example, method 1000optionally includes one or more of the characteristics of the variousmethods described above/below with reference to methods 800 and 1200.For example, the computer systems can be the same systems. For anotherexample, the audio message sent in method 800 can be the media receivedin method 1000. For another example, the voice input transmitted inmethod 1200 can be the same as the media received in method 1000. Forbrevity, these details are not repeated below.

FIGS. 11A-11R illustrate electronic devices and exemplary userinterfaces for managing visual displays. The user interfaces in thesefigures are used to illustrate the processes described below, includingthe processes in FIG. 12.

FIGS. 11A-11R illustrate scenarios where device 1100 displays a visualindication with visual content of a first type (e.g., single-coloredlights) when device 1100 determines that a portion of the voice inputshould be transmitted to one or more external devices as an audiomulticast message and where device 1100 displays a visual indicationwith visual content of a second type (e.g., multi-colored lights) whendevice 1100 cannot determine that a portion of the voice input should betransmitted to one or more external devices as an audio multicastmessage (e.g., determines that no portion of the voice input should betransmitted to one or more external devices as an audio multicastmessage). The first and second types of visual content are describedbelow for exemplary purposes only. The first and second types of visualcontent are different to provide a visual indication to the user ofwhether uttered voice input will be potentially multicast to one or moreexternal devices.

FIG. 11A illustrates electronic device 1100 in a position on a tablethat is located in front of a person. Device 1100 is a smart speaker andincludes one or more features of device 580. For example, device 1100includes one or more displays 1102 that includes one or more componentsof one or more displays 580E. Displays 1102 include a plurality of LEDs.In some embodiments, device 1100 includes one or more features of device100, 300, or 500. In FIG. 11A, device 1100 is associated with home 602and living room 602 d for exemplary purposes only. Thus, in FIG. 11A,device 1100 is one of living room speakers 646 and 648, as describedabove in relation to FIG. 6. In some embodiments, device 1100 isassociated with another room in home 602 (or another home).

As illustrated in FIG. 11A, device 1100 is in an inactive state (e.g., asleep state, a low power state, a state where device 1100 is notdisplaying, via displays 1102, visual content), such that displays 1102are not displaying visual content (e.g., a visual indication). In someembodiments, when device 1100 is not displaying visual content, the LEDsof displays 1102 are not lit.

FIGS. 11A-11L illustrate device 1100 displaying visual content inresponse to receiving voice input that includes information that is tobe multicast to one or more target group of devices. In someembodiments, one or more similar techniques described above in relationto device 700 receiving voice input FIGS. 7M-7N and 7S-7Z can be appliedto device 1100 receiving voice input, as described below (orvice-versa).

FIGS. 11B-11F illustrate device 1100 receiving voice input from theperson standing in front of device 1100. The voice input received bydevice 1100 in FIGS. 11B-11F is “HEY ASSISTANT, MULTICAST, WHAT MUSIC DOYOU WANT TO PLAY IN THE LIVING ROOM?” The voice input received in FIGS.11B-11F can be received in a natural speaking way or another way. FIGS.11B-11F are illustrated to provide an exemplary embodiment of how device1100 may react to different portions of the voice input.

At FIG. 11B, device 1100 receives, via one or more microphones of device1100, voice input portion 1150 b from the person standing in front ofdevice 1100. Here, the person has uttered the phrase, “HEY ASSISTANT.”

As illustrated in FIG. 11B, in response to receiving voice input portion1150 b, device 1100 changes to be in an active state (e.g., an awakestate, a powered state, a state where displays 1102 are displayed visualcontent), such that device 1100 displays, via displays 1102, visualcontent. In FIG. 11B, device 1100 displays visual content (e.g., avisual indication) (e.g., lights) that is multi-colored, which isrepresented by the different types of discs on displays 1102 (e.g., theblack discs, white discs, and patterned discs of 1102 in FIG. 11B). AtFIG. 11B, device 1100 displays visual content that is multi-coloredbecause device 1100 has not determined that a portion of the voice inputshould be transmitted to one or more external devices as an audiomulticast message.

At FIG. 11C, device 1100 receives, via one or more microphones of device1100, voice input portion 1150 c from the person standing in front ofdevice 1100. Here, the person has uttered the phrase, “MULTICAST.”

As illustrated in FIG. 11C, in response to receiving voice input portion1150 c, device 1100 remains in the active state. In FIG. 11C, device1100 displays visual content that is a single color (e.g., the blackdiscs of 1102 in FIG. 11B without there being any other types of discs)because device 1100 has determined that a portion of the voice inputshould be transmitted to one or more external devices as an audiomulticast message. Here, device 1100 determines that the portion of thevoice input should be transmitted (e.g., multicast) as an audiomulticast message because device 1100 detected an instruction (e.g.,“MULTICAST”) phrase that indicates that the person is issuing amulticast command (e.g., a command to multicast an audio message).

At FIG. 11D, device 1100 receives, via one or more microphones of device1100, voice input portion 1150 d from the person standing in front ofdevice 1100. Here, the person has uttered the phrase, “WHAT MUSIC.”

As illustrated in FIG. 11D, in response to receiving voice input portion1150 d, device 1100 remains in the active state. In FIG. 11D, device1100 continues to display visual content that is a single color based onthe determination that a portion of the voice input should betransmitted to one or more external devices as an audio multicastmessage (e.g., as described above in relation to FIG. 11C). In someembodiments, device 1100 determines that a portion of the voice inputshould be transmitted to one or more devices assigned to home 602 (e.g.,for similar reasons as described above in relation to FIGS. 7M-7N). Insome embodiments, device 1100 determines the one or more devices totransmit a portion of the voice input based on whether a portion of thevoice input is assigned to a group of devices based on the voice inputcorresponds to a group identifier (e.g., “Home,” “Living Room,”“Entertainment Zone,” “to Dining Room”).

Notably, when compared to FIG. 11C, the pattern of the visual contentdisplayed by device 1100 in FIG. 11D is different from the pattern ofthe visual content displayed by device 1100 in FIG. 11C. As illustratedin FIGS. 11C-11D, device 1100 displays, via displays 1102, differentvisual content that reacts to the voice of the person. In other words,because “MULTICAST” and “WHAT MUSIC” are different phrases, device 1100changes the pattern of the visual content. In some embodiments, device1100 changes the pattern of the visual content based on the speed,sharpness, volume, pitch, tone, and/or rhythm (e.g., beat) of theportion of the voice content, such that portions of voice content thatare different produce a different pattern of visual content. In someembodiments, device 1100 changes the color saturation, brightness, size,or position of the visual content based on the speed, sharpness, volume,pitch, tone, and/or rhythm (e.g., beat) of the portion of the voicecontent. In some embodiments, device 1100 does not change the pattern ofthe visual content and/or is not reactive to the voice of the person.

At FIG. 11E, device 1100 continues to receive, via one or moremicrophones of device 1100, voice input portion 1150 e from the personstanding in front of device 1100. Here, the person has uttered thephrase, “DO YOU WANT TO PLAY.”

As illustrated in FIG. 11E, in response to receiving voice input portion1150 e, device 1100 remains in the active state. In FIG. 11E, device1100 continues to display visual content that is a single color based onthe determination that a portion of the voice input should betransmitted to one or more external devices. In addition, device 1100changes the pattern of visual content because voice input portion 1150 eis different from voice input portion 1150 d.

At FIG. 11F, device 1100 continues to receive, via one or moremicrophones of device 1100, voice input portion 1150 f from the personstanding in front of device 1100. Here, the person has uttered thephrase, “IN THE LIVING ROOM?”

As illustrated in FIG. 11F, in response to receiving voice input portion1150 f, device 1100 remains in the active state. In FIG. 11F, device1100 continues to display visual content that is a single color based onthe determination that a portion of the voice input should betransmitted to one or more external devices. In addition, device 1100changes the pattern of visual content because voice input portion 1150 fis different from voice input portion 1150 e.

At FIG. 11F, device 1100 detects an end of the voice input received inFIGS. 11B-11F. At FIG. 11F, in response to detecting the end of thevoice input, device 1100 sends one or more portions of the voice inputto one or more external devices as an audio multicast message. In someembodiments, the audio multicast message includes 1150 c-1150 d (e.g.,“WHAT MUSIC DO YOU WANT TO PLAY IN THE LIVING ROOM?”). In someembodiments, the audio multicast message does not include voice inputportions 1150 b (“HEY, ASSISTANT”) and/or 1150 c (“MULTICAST”) becauseof a determination that voice input portions 1150 b and/or 1150 ccorrespond to a command portion of the voice input. In some embodiments,when the input includes a portion that corresponds to a target groupidentifier (e.g., “Home” or “Living Room”), the target group identifier(not the portion of the voice input that corresponds to the target groupidentifier) is transmitted along with the audio multicast message (e.g.,as plain text) but is not output as audio by the one or more externaldevices. In some embodiments, the audio multicast message that is outputby the one or more external devices sounds like the person (is arecording that is played back of the person who uttered the voice input)and is not computer-generated speech via a device.

As illustrated in FIG. 11G, device 1100 provides (e.g., outputs) output1154 g, which indicates that a portion of the voice input has been sentto the target group identifier that corresponds to home 602 (e.g., “123Main St.” as described above in relation to home location identifier712). When outputting output 1154 g, device 1100 displays visual contentthat is a single color because of the determination that a portion ofthe voice input (e.g., received in FIGS. 11B-11F) should be transmittedto one or more external devices as an audio multicast message. In someembodiments, device 1100 displays multi-colored visual content whenoutputting audio confirmation 1154 g, irrespective of thisdetermination.

As illustrated in FIG. 11H, device 1100 changes to an inactive state. AtFIG. 11I, device 1100 receives data corresponding to an audio multicastmessage from Jane, via one or more of the external devices assigned tohome 602.

As illustrated in FIG. 11I, in response to receiving the audio multicastmessage (and without (independent of) receiving user input), device 1100provides output 1154 i, which is an introductory tone or chime. Asillustrated in FIG. 11J, after providing output 1154 i, device 1100provides output 1154 j, which corresponds to the audio multicast messagethat was received at FIG. 11I.

As illustrated in FIGS. 11I-11J, device 1100 displays visual contentthat is the single color when providing output 1150 i and output 1150 j.Device 1100 displays visual content that is a single color becausedevice 1100 is outputting an audio multicast message. As illustrated inFIGS. 11I-11J, device 1100 also displays visual content that is the samepattern. Thus, in FIGS. 11I-11J, device 1100 does not change the visualcontent based on the different content of outputs 1150 i and output 1150j. In some embodiments, device 1100 does change the visual content basedon the different content of output 1150 i and output 1150 j. In someembodiments, device 1100 provides output 11541150 i and/or output 1150 jat the current volume level (e.g., system volume level) of the device1100. In some embodiments, one of output 1154 i and/or output 1150 j areprovided at the volume level of device 1100 while the other one is not.In some embodiments, device 1100 receives multiple audio multicastmessages within a period of time and outputs the same (or different)introductory tone before each of the audio multicast messages is played.In some embodiments, device 1100 receives multiple audio multicastmessages within a period of time and outputs the same (or different)introductory tone before each group (e.g., grouped by target group ofdevices) of audio multicast messages that are played.

As illustrated in FIG. 11J, the audio message provided by device 1100says, “JANE REPLIED FROM HOME: HOW ABOUT JAZZ?” Here, the audio messageprovided by device 1100 includes a target identifier; in someembodiments, it does not.

At FIG. 11K, device 1100 receives, via one or more microphones of device1100, voice input portion 1150 k from the person standing in front ofdevice 1100. Here, the person has uttered the phrase, “HEY ASSISTANT,REPLY, YOU GOT IT.”

As illustrated in FIG. 11K, device 1100 displays visual content that isa single color (e.g., the black discs of 1102 in FIG. 11K) becausedevice 1100 has determined that a portion of voice input portion 1150 kshould be transmitted to one or more external devices as an audiomulticast message. In some embodiments, device 1100 makes thisdetermination because voice input portion 1150 k was received within apredetermined time (e.g., 1, 2, 30, 60 seconds) after device 1100provided output 1150 j (e.g., audio multicast message received in FIG.11J). In some embodiments, device 1100 makes this determination becausevoice input portion 1150 k includes the phrase, “Reply,” and/or includesthe phrase, “Reply,” within a second predetermined amount of time afterdevice 1100 provided output 1150 j. In some embodiments, the secondpredetermined period of time is greater than the first predeterminedperiod of time.

As illustrated in FIG. 11L, device 1100 changes to an inactive state.

FIGS. 11M-11R illustrate device 1100 displaying visual content inresponse to receiving voice input that does not include information thatis to be multicast to one or more target group of devices.

FIGS. 11M-11O illustrated device 1100 receiving voice input from theperson standing in front of device 1100. The voice input received bydevice 1100 in FIGS. 11M-11O is “HEY ASSISTANT, PLAY JAZZ MUSIC IN THELIVING ROOM.” The voice input received in FIGS. 11M-11O can be receivedin a natural speaking way or another way. FIGS. 11M-11O are illustratedto provide an exemplary embodiment of how device 1100 may react todifferent portions of the voice input.

At FIG. 11M, device 1100 receives, via one or more microphones of device1100, voice input portion 1150 m from the person standing in front ofdevice 1100. Here, the person has uttered the phrase, “HEY ASSISTANT.”As illustrated in FIG. 11M, in response to receiving voice input portion1150 m, device 1100 changes to an active state, using one or moretechniques similar to those described above in FIG. 11B. In addition,device 1100 displays visual content (e.g., lights) that is multi-coloredbecause device 1100 has not determined that a portion of the voice inputshould be transmitted to one or more external devices as an audiomulticast message.

At FIG. 11N, device 1100 receives, via one or more microphones of device1100, voice input portion 1150 n from the person standing in front ofdevice 1100. Here, the person has uttered the phrase, “PLAY JAZZ MUSIC.”As illustrated in FIG. 11N, in response to receiving voice input portion1150 n, device 1100 displays visual content (e.g., lights) that ismulti-colored because device 1100 has not determined that a portion ofthe voice input should be transmitted to one or more external devices asan audio multicast message.

At FIG. 11O, device 1100 receives, via one or more microphones of device1100, voice input portion 1150 o from the person standing in front ofdevice 1100. Here, the person has uttered the phrase, “IN THE LIVINGROOM.” As illustrated in FIG. 11O, in response to receiving voice inputportion 1150 o, device 1100 displays visual content (e.g., lights) thatis multi-colored because device 1100 has not determined that a portionof the voice input should be transmitted to one or more external devicesas an audio multicast message.

In FIGS. 11M-11O, device 1100 the visual content is not reactive to thevoice input. In some embodiments, device 1100 does not change thepattern of the visual content when device 1100 has not determined that aportion of the voice input should be transmitted to one or more externaldevices as an audio multicast message. In some embodiments, device 1100changes the pattern of the visual content to be reactive to differentvoice input portions, irrespective of this determination.

As illustrated in FIG. 11P, device 1100 provides (e.g., outputs) output1154 p, which indicates that device 1100 is now playing jazz radio. Whenproviding output 1154 g, device 1100 displays visual content (e.g.,visual indication) that is multi-colored because of the determinationthat a portion of the voice input (e.g., received in FIGS. 11M-11O)should not be transmitted to one or more external devices as an audiomulticast message. At FIG. 11P, device 1100 also outputs jazz music.

As illustrated in FIG. 11Q, while outputting jazz music, device 1100changes to an inactive state.

As illustrated in FIG. 11R, device 1100 receives, via one or moremicrophones of device 1100, voice input portion 1150 r from the personstanding in front of device 1100. Here, the person has uttered thephrase, “PAUSE.” As illustrated in FIG. 11R, in response to receivingvoice input portion 1150 r, device 1100 displays visual content (e.g.,lights) that is multi-colored because device 1100 has not determinedthat a portion of the voice input should be (e.g., has determined thatno portion of the voice input should be) transmitted to one or moreexternal devices as an audio multicast message. At FIG. 11R, device 1100also stops outputting the jazz music and pauses the media correspondingto the jazz music, in response to receiving the “PAUSE” instruction.

FIG. 12 is a flow diagram illustrating a method for displaying visualindications using an electronic device in accordance with someembodiments. Method 1200 is performed at a device (e.g., 100, 300, 500,634, 674, 700, 701, 850) having one or more microphones, one or morespeakers, and one or more display devices. Some operations in method1200 are, optionally, combined, the orders of some operations are,optionally, changed, and some operations are, optionally, omitted.

As described below, method 1200 provides an intuitive way for displayingvisual indications. The method reduces the cognitive burden on a userfor determining a type of command given to a device, thereby creating amore efficient human-machine interface. For battery-operated computingdevices, enabling a user to recognize the type of command faster andmore efficiently conserves power and increases the time between batterycharges.

The electronic device receives (1202) voice input (e.g., 1150 a-1150 r)via the one or more microphones.

The electronic device displays (1204) a visual indication (e.g., lightsdisplayed via 1102) of the voice input, where: in accordance with adetermination that a portion of the voice input (e.g., 1150 b-1150 f,1150 r) corresponds to voice input that is to be transmitted to one ormore devices (e.g., and/or transmitted (e.g., multicast) to a pluralityof devices) (e.g., and, in some embodiments, another portion (e.g., aportion of the voice input that is not transmitted) of the voice inputcorresponds to a first type of command (e.g., command that initiates aprocess for transmitting at least some of the voice input to a pluralityof external devices (e.g., personal device (e.g., cellular phone),external accessory devices (e.g., smart home accessory device (e.g., asmart (e.g., intelligent) light bulbs, thermostat, door lock, dooropener (e.g., garage door opener), speaker, television, electric outlet,power strip, camera, appliance (e.g., a refrigerator, stove))))),displays (1206), via the one or more display devices, the visualindication (e.g., 1102 in 11B-11F, 11K) includes displaying the visualindication with a first set of one or more colors; and in accordancewith a determination that the voice input (e.g., 1150 m-1150 o, 1150 r)does not include an instruction to transmit (e.g., an instruction tomulticast) any portion of the voice input to other devices (e.g., tooutput a part of the voice input, such as by speakers of the otherdevices (e.g. a plurality of other devices)) (e.g., another portion ofthe voice input is a second type of command), displays (1210), via theone or more display devices, the visual indication (e.g., 1102 in FIGS.11M-11O, 11R) includes displaying the visual indication with a secondset of one or more colors that is different from the first set of one ormore colors. In some embodiments, the first type of command is a commandthat initiates a process for transmitting at least some of the voiceinput (e.g., the portion of the voice input that is to be transmitted)to a plurality of external devices. In some embodiments, transmitting atleast some of the voice input to the plurality of devices causes two ormore of the plurality of external devices to output (e.g., automatically(e.g., without intervening user input) output) some of the voice input.In some embodiments, the plurality of external devices includesdifferent types (e.g., a personal device, an external accessory device)of external electronic device. In some embodiments, the plurality ofdevices are associated with a first group that is associated with atop-level location (or destination) (e.g., a location of a first type)(e.g., a home location, business, an office (e.g., a location that is atop-level location (e.g., a location that defines one or more locations(e.g., sub-locations) and/or one or more regions (or zones) that arewithin an area and/or perimeter that is defined by the location of thetop-level location, a location that is at the top level of a hierarchyof locations, a location that is linked (e.g., programmatically) to oneor more locations (e.g., sub-locations) and/or one or more regions (orzones)))). In some embodiments, the first group of devices can include aplurality of subgroups of devices. In some embodiments, a sub-group ofdevices is associated with a sub-location (e.g., a room positioned(e.g., in the boundary, area, or perimeter of the top-level location;and/or defined, identified, and/or linked (e.g., programmaticallylinked) (e.g., by a user) to the top-level location) in a top-levellocation and/or a zone (e.g., a location that is defined by two or moresub-locations within (e.g., within the boundary, area, or perimeter ofthe top-level location and/or defined, identified, and/or linked (e.g.,by a user)) the top-level location (e.g., an entertainment zone thatincludes a living room and a dining room within the top-level location,a personal zone that includes two or more bedrooms, living rooms,bathrooms within the top-level location) in the top-level location). Insome embodiments, the first user interface is a control user interface.In some embodiments, the plurality of devices is associated with aplurality of subgroups of devices. In some embodiments, the visualindication continues to be displayed while receiving voice input via theone or more microphones. In some embodiments, the visual indicationceases to displays after initially being displayed while receiving voiceinput via the one or more microphones. In some embodiments, the secondtype of command is not a command that initiates a process fortransmitting at least some of the voice input to a plurality of externaldevice (e.g., speakers, personal device (e.g., cellular phone). In someembodiments, the visual indication continues to be displayed whilereceiving voice input via the one or more microphones. In someembodiments, the visual indication ceases to displays after initiallybeing displayed while receiving voice input via the one or moremicrophones. Displaying a visual indication with a first set of colorswhen it is determined that a portion of the voice input is to betransmitted to one or more devices provides the user with feedbackregarding whether a portion of the voice input will be played back atexternal electronic devices, such that a user can be aware that theirutterances can be potentially heard by other people. Providing improvedvisual feedback to the user enhances the operability of the device andmakes the user-device interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently. Displaying a visualindication with a second set of one or more colors (e.g., that isdifferent from the first set of one or more colors) when it isdetermined that that the user's voice input does not include aninstruction to transmit any portion of the voice input to other devicesprovides the user with reassurances that the user's utterances will notbe heard by individuals at other external devices. Providing improvedvisual feedback to the user enhances the operability of the device andmakes the user-device interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some embodiments, voice input (e.g., 1150 m-1150 o) requesting forthe electronic device to retrieve information (e.g., retrieveinformation, such as the weather, answers to one or more queries) orchange one or more states (e.g., change one or more of an audio state(e.g., answering/placing a phone call, adjusting a volume), a state ofmedia playback (e.g., pausing, playing, fast-forwarding, rewinding,skipping, shuffling media), an on/off state) of the electronic device isthe voice input that does not include an instruction to transmit anyportion of the voice input to other devices.

In some embodiments, the voice input that corresponds (e.g., 1150 b-1150f, 1150 k) to voice input that is to be transmitted to the one or moredevices includes (1208) an identifier (e.g., a location, a sub-location,a location that corresponds to a group (“home”) or category (e.g.,“dining room,” “living room,” “sunroom” within the group) for a group ofdevices. In some embodiments, the voice input includes the identifierfor the group of devices and an audio message that will be transmittedto the group of devices. In some embodiments, the voice input (e.g.,1150 m-1150 o, 1150 r) that does not include an instruction to transmitany portion of the voice input to other devices does not include (1212)the identifier for the group of devices. In some embodiments, the voiceinput does not include an identifier for a group of devices and an audiomessage is not transmitted to the group of devices.

In some embodiments, the first set of one or more colors (e.g., 1102 inFIGS. 11C-11F) is a solid color and the second set of one or more colors(e.g., 1102 in FIGS. 11M-11O) is a plurality of colors. In someembodiments, the solid color is not included in the plurality of colors.Applying different visual effects to the electronic device whendifferent prescribed conditions are met allows the user to quicklyrecognize whether their utterances will be transmitted and played backby external electronic devices. Providing improved visual feedback tothe user enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some embodiments, as a part of displaying the visual indication(e.g., 1102 in FIGS. 11C-11F) with the first set of one or more colors,the electronic device displays an animation (e.g., 1102 in FIGS.11C-11F) (e.g., of a pattern (e.g., a spatial pattern) of lights) thatchanges (e.g., in size, brightness, saturation, and/or position) over aperiod of time (e.g., 1, 5, 10 seconds) (e.g., based on the amount oftime that the voice input is received) based on the voice input (e.g.,based on a variation in a value for an input parameter (such as avolume, tone, and/or pitch) of the voice input).

In some embodiments, as a part of displaying the animation (e.g., 1102in FIGS. 11C-11F), the electronic device changes (e.g., increasing ordecreasing) one or more of the brightness, size, color saturation, orposition of the visual indication (e.g., 1102 in FIGS. 11C-11F) based onone or more of a volume or pitch of the voice input. In someembodiments, as the volume and/or pitch of the voice inputincreases/decreases the one or more of the brightness, size, colorsaturation of the visual indication. In some embodiments, as the rhythm(e.g., or beat) of the voice input increases/decreases the rhythm (orbeat) of the animation increases/decreases. In some embodiments, as thevolume and/or pitch of the voice input increases/decreases the visualindication takes up more/less of the area of the display device.Updating the visual characteristics (e.g., brightness, size, colorsaturation, or position) of the visual indication to reflect a change inthe user's voice input (e.g., change in volume or pitch of voice input)provides the user with visual feedback regarding various characteristicsof the user's voice input. Providing improved visual feedback to theuser enhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

In some embodiments, the electronic receives data corresponding to aplurality of audio messages for playback. In some embodiments, the datacorresponding to the plurality of audio messages for playback includesan identifier (e.g., a location that corresponds to a group (“home”) orcategory (e.g., “dining room,” “living room,” “sunroom” within thegroup) for a group of devices. In some embodiments, the plurality ofaudio messages are for a plurality of identifiers, where each identifiercorresponds to a different group or category. In some embodiments, inresponse to receiving the data corresponding to the plurality of audiomessage for playback, the electronic device outputs an introductory tone(e.g., 1154 i) (e.g., a dinging or ringing sound, the introductory toneor chime for all audio messages) before (e.g., and before outputting anidentifier for the group of devices) outputting each respective audiomessage of the plurality of audio messages. In some embodiments, theidentifier is output before a group of audio messages that correspondsto the identifier (e.g., that identifies a target group of devices) areoutput. Outputting same introductory tone before outputting differentaudio messages that have been multicast alerts recipients that the audiomessage is a multicast type of audio message. Providing improved audiofeedback enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some embodiments, the electronic device receives data correspondingto an audio message (e.g., an audio multicast message) for playback. Insome embodiments, the data corresponding to the audio message forplayback includes an identifier (e.g., a location that corresponds to agroup (“home”) or category (e.g., “dining room,” “living room,”“sunroom” within the group) associated with a group of devices. In someembodiments, in response to receiving the data, the electronic device,automatically, without user input, outputs (e.g., 1154 j) (e.g., via aspeaker of the electronic device) the audio message. In someembodiments, outputting the audio message includes outputting anidentifier (e.g., a location that corresponds to a group (“home”) orcategory (e.g., “dining room,” “living room,” “sunroom” within thegroup) for a group of devices.

In some embodiments, before outputting the audio message, displaying asecond visual indication (e.g., 1102 in 11I) in response to receivingthe data. In some embodiments, the second visual indication has a firstvisual appearance (e.g., color, size, brightness) when the audio messageis an audio message transmitted to multiple devices (e.g., a group ofdevices). In some embodiments, the second visual indication has a secondvisual appearance that is different from the first visual appearancewhen the audio message is an audio message transmitted to the electronicdevice and not to other devices.

In some embodiments, the electron device receives data corresponding toan audio message (e.g., an audio multicast message) for playback. Insome embodiments, the data corresponding to the audio message forplayback includes an identifier (e.g., a location that corresponds to agroup (“home”) or category (e.g., “dining room,” “living room,”“sunroom” within the group) associated with a group of devices. In someembodiments, outputs the audio message (e.g., automatically, in responseto a user request). In some embodiments, while outputting the audiomessage, the electronic device concurrently displays a visual indication(e.g., 1102 in FIG. 11J) that does not change based on one or morecharacteristics (e.g., volume, pitch, tone, rhythm (e.g., beat)) of theaudio message.

In some embodiments, the audio message is output (e.g., 1154 j) based onthe volume level (e.g., a system-level audio level) (e.g., a currentvolume level) of the electronic device. In some embodiments, theelectronic device is not displaying visual content (e.g., 1102 in FIG.11A) (e.g., any lights (e.g., no LEDs are turned on)) before the visualindication (e.g., 1102 in FIGS. 11B-11F) is displayed. In someembodiments, the electronic device is in an inactive display state(e.g., LEDs are not lit) before the visual indication is displayed. Insome embodiments, while displaying the visual indication, the electronicdevice is in an active display state (e.g., some LEDs are lit). Notdisplaying visual content when no voice input is being received providesthe user with visual feedback that no voice input is being received andthat any utterances by the user are not being transmitted and playedback by other external devices. Providing improved visual feedback tothe user enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some embodiments, the electronic device ceases (1214) to display thevisual indication (e.g., 1102 in FIG. 11G), including ceasing to displayvisual content (e.g., 1102 in FIG. 11H) via the one or more displaydevices. In some embodiments, the electronic device is in an inactivedisplay state after ceasing to display the visual content. Notdisplaying visual content when no voice input is being received providesthe user with visual feedback that no voice input is being received andextends the battery life of the device because the device is in aninactive state. Providing improved visual feedback to the user enhancesthe operability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

In some embodiments, the visual indication (e.g., 1102 in FIG. 11G) isnot displayed concurrently with another visual indication (e.g., avisual indication that corresponds to a status or a process that isdifferent from the status or process that the visual indicationcorresponds to (e.g., a multicast process or a process forsending/receiving audio multicast messages)) (e.g., or other visualcontent). Providing only one visual indication at a time provides theuser with clarity regarding the status of the electronic device andwhether the user's utterances are being transmitted and played back byother external devices. Providing improved visual feedback to the userenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

In some embodiments, the visual indication (e.g., 1102 in FIG. 11G) is alight (e.g., LEDs of 1102) indication that provides different lightsbased on a status (e.g., a status for a multicast process or a processfor sending/receiving audio multicast messages, a process associatedwith a state of the electronic device, a process associated with a stateof the electronic device) of one or more applications in communicationwith (e.g., executing on the electronic device or one or more externalapplications that send data to or receive data from the electronicdevice) the electronic device.

Note that details of the processes described above with respect tomethod 1200 (e.g., FIG. 12) are also applicable in an analogous mannerto the methods described above. For example, method 1200 optionallyincludes one or more of the characteristics of the various methodsdescribed above with reference to methods 800 and 1000. For example, thecomputer systems can be the same systems. For another example, the audiomessage sent in method 800 can be the media received in method 1000. Foranother example, the voice input transmitted in method 1200 can be thesame as the media received in method 1000. For brevity, these detailsare not repeated below.

FIGS. 13A-13N illustrate exemplary electronic devices and userinterfaces for managing external devices in accordance with someembodiments. The user interfaces in these figures are used to illustratethe processes described below, including the processes in FIG. 14.

FIG. 13A illustrates electronic device 700 displaying user interface710. In some embodiments, device 700 can be related to (e.g., incommunication with or not in communication with one or more networks inrelation to, positioned at a physical location relative to, interactwith one or more devices assigned to, etc.) home 602 as described above(e.g., in FIGS. 7A-7AH). User interface 710 is a user interface forcontrolling multiple external devices assigned to home 602. Userinterface 710 includes home location identifier 712, macro controls 716,external device controls 718, navigations controls 722, home multicastinstruction 724, and home multicast affordance 726 (e.g., selectableuser interface object), which have one or more characteristics and canbe responded to in a manner as described above in relation to FIGS.7A-7AH.

User interface 710 also includes status indicators 1314. Each of thestatus indicators includes information concerning the current status ofsome of the devices currently assigned to home 602 (e.g., some of whichhave been described above in relation to FIG. 6). As illustrated in FIG.13A, status indicators 1314 include carbon monoxide status indicator1314 a, alarm status indicator 1314 b, account error status indicator1314 c, update available status indicator 1314 d, and a portion of timerstatus indicator 1314 e (respectively, from left-to-right). Carbonmonoxide status indicator 1314 a indicates that one or more carbonmonoxide detectors or sensors have detected the presence of carbonmonoxide in home 602 and, thus, the one or more carbon monoxidedetectors or sensors are in a carbon monoxide detected state, e.g.,alert “on” state, detecting a carbon monoxide leak. Alarm statusindicator 1314 b indicates that one or more alarms have been trippedthat are related to home 602 and, thus, the one or more alarms are in anactive, e.g., triggered or alerting, state. Account error statusindicator 1314 c indicates that device 700 is currently having issueswith an account associated with home 602, such as device 700 not beingcurrently logged into the account associated with home 602 or device 700having issues connecting to one or more known external devices assignedto home 602. Update available status indicator 1314 d indicates thatupdated software is available to be installed on one or more externaldevices and/or device 700 (e.g., software that contains one or morefunctions corresponding to the home application). Timer status indicator1314 e indicates that one or more timers are currently active. Inaddition, timer status indicator 1314 e indicates that at least onetimer (or an aggregate of the total time of multiple timers) is in astate where about 0% of the time has elapsed from the time that thetimer (or multiple timers) was originally set. In particular, this stateis represented by remaining time indication 1314 e 1, which coversapproximately 100% of the perimeter of timer status indicator 1314 e.

Each status indicator contains a name (e.g., “carbon monoxide)associated with the status indicator (e.g., carbon monoxide statusindicator 1314 a) and can be displayed with at least one particularvisual characteristic (e.g., a color (e.g., red, greyed-out, etc.), apattern, a shape, etc.). In some embodiments, one or more of the statusindicators include a representation of a number of devices of a certaintype (e.g., carbon monoxide detector or sensor) that are in the state(e.g., active, alerting, or needing attention) that is represented bythe status indicators. Each status indicator may also include a quickvisual representation of the status indicator's state.

In FIG. 13A, each status indicator is within (e.g., included, a part of,or belongs to) a particular status indicator category, and the statusindicators are displayed in order based on the respective statusindicator category to which each of the respective status indicatorscorresponds. In FIG. 13A, the particular status indicator category of astatus indicator denotes the urgency (or importance) of the statusindicator. Status indicators that are within one status indicatorcategory can have one or more characteristics that are different fromthe one or more characteristics of status indicators that are within adifferent status indicator category. As illustrated in FIG. 13A, carbonmonoxide status indicator 1314 a and alarm status indicator 1314 b arein a first status indicator category (e.g., a most urgent category, analarm category) and are displayed with a first color (e.g., red, asindicated by the vertical hatching). In some embodiments, the firststatus indicator category indicates that the external devices associatedwith a particular status indicator should be addressed as soon aspossible. Next, in FIG. 13A, account error status indicator 1314 c andupdate available status indicator 1314 d are in a second statusindicator category (e.g., a second-most urgent category, a statusindicator category that is less urgent than the first status indicatorcategory, an action required category) and are displayed with adifferent color (e.g., as indicated by no hatching) from the color ofthe status indicators in the first category. In FIG. 13A, timer statusindicator 1314 e is in a third status indicator category (e.g., athird-most urgent category, a status indicator category that is lessurgent than the second status indicator category). Timer statusindicator 1314 e includes one or more dynamic components (e.g.,remaining time indication 1314 e 1) that are relevant to the state ofthe timer that corresponds to timer status indicator 1314 e. In someembodiments, the one or more dynamic components (e.g., remaining timeindication 1314 e 1) are updated in real-time (e.g., as furtherexplained below with reference to FIG. 13B). In some embodiments, statusindicators that are in the first and/or second status indicatorcategories do not have one or more dynamic components that are relativeto the particular state of the one or more external devices thatcorrespond to the respective status indicator.

In some embodiments, when a determination is made that a new statusindicator will need to be displayed, device 700 will display the newstatus indicator after (or before or among) the status indicators in thesame status indicator category as the new status indicator. For example,when a determination is made that a new status indicator should bedisplayed and the new status indicator is in the second status indicatorcategory, device 700 displays the new status indicator between updateavailable status indicator 1314 d and timer status indicator 1314 e(e.g., instead of appending the new status indicator at the beginningand/or end of all of the displayed status indicators 1314). Thus, thenew status indicator is displayed on device 700 at a position that isrelevant to the urgency of the status indicator to allow the user toidentify and/or interact with the new status indicator based on the newstatus indicator's relevant urgency as compared to the relevant urgencyof the other displayed status indicators. At FIG. 13A, device 700detects gesture 1350 a at a location that corresponds to statusindicators 1314.

At FIG. 13B, device 700 determines that gesture 1350 a is a leftwardswipe gesture. As illustrated in FIG. 13B, because device 700 hasdetermined that gesture 1350 a is a swipe gesture, device 700 displaysadditional status indicators if there are more status indicators thancan be displayed (e.g., within the same region as status indicators1314) on device 700 at one time. In particular, device 700 moves statusindicators 1314 that were previously displayed on FIG. 13A to the left(e.g., in the direction of gesture 1350 a), such that carbon monoxidestatus indicator 1314 a, alarm status indicator 1314 b, and accounterror status indicator 1314 c cease to be displayed, update availablestatus indicator 1314 d and timer status indicator 1314 e haverespectively replaced carbon monoxide status indicator 1314 a and alarmstatus indicator 1314 b, and climate status indicator 1314 f, door lockstatus indicator 1314 g, and a portion of light status indicator 1314 hare newly displayed at FIG. 13B.

As illustrated in FIG. 13B, timer status indicator 1314 e includes adynamic component, remaining time indication 1314 e 1. When looking atFIG. 13A-13B, remaining time indication 1314 e 1 moves (e.g., not inresponse to user input but rather in response to time passing) frombeing around roughly 100% of the perimeter of timer status indicator1314 e in FIG. 13A to being around roughly 75% of the perimeter of timerstatus indicator 1314 e in FIG. 13B since the timer starts at 13 minutesin FIG. 13A and runs until it has approximately 10 minutes left in FIG.13B.

In FIG. 13B, climate status indicator 1314 f is in the third statusindicator category, which is the same status category that timer statusindicator 1314 e is in as discussed above in relation to FIG. 13A.Climate status indicator 1314 f represents a status indicator thatcorresponds to the states of multiple different types (e.g., air qualitysensors, temperature, humidity sensors) of external devices. Climatestatus indicator 1314 f indicates that an average temperature detectedby one or more climate control devices (or in which one or more climatecontrol devices are set) is 68 degrees, the average humidity detected byone or more humidity sensors is 40%, and the average air qualitydetected by one or more air quality sensors is good. Moreover, statusindicator 1314 f also indicates that the thermostat control is causing aheating system to be activated (e.g., a room is being heated by a devicethat the thermostat control is controlling), which is indicated by theupward point arrow on status indication 1314 f. In some embodiments,when status indicator 1314 f includes a downward pointing arrow, statusindicator 1314 f indicates that the thermostat control is causing an airconditioning system to be activated (e.g., a room is being cooled by adevice that the thermostat control is controlling). In some embodiments,when status indicator 1314 f includes a dash, status indicator 1314 findicates that the thermostat control is not causing an air-conditioningsystem or a heating system to be activated (e.g., a room is detectedbeing at the specified temperature to which the thermostat is set). AtFIG. 13B, device 700 detects gesture 1350 b at a location thatcorresponds to status indicators 1314.

At FIG. 13C, device 700 determines that gesture 1350 b is a leftwardswipe gesture. As illustrated in FIG. 13C, because device 700 hasdetermined that gesture 1350 b is a swipe gesture, device 700 displaysadditional status indicators. In particular, device 700 moves statusindicators 1314 that were previously displayed on FIG. 13B to the left,such that update available status indicator 1314 d, timer statusindicator 1314 e, and climate status indicator 1314 f cease to bedisplayed, door lock status indicator 1314 g, light status indicator1314 h has slid to the left, and open window status indicator 1314 i,door sensor status indicator 1314 j, and motion sensor status indicator1314 k are newly displayed at FIG. 13C. In embodiments where gesture1350 b is determined to be a rightward swipe gesture, device 700 movesthe status indicators to right to display one or more of the previouslydisplayed status indicators (e.g., carbon monoxide status indicator 1314a, alarm status indicator 1314 b displayed in FIG. 13A).

At FIG. 13C, door lock status indicator 1314 g, light status indicator1314 h, and open window status indicator 1314 i are in a fourth statusindicator category (e.g., a fourth-most urgent category, a statusindicator category that is less urgent than the third status indicatorcategory, a quick action status indicator category). Although, asdescribed, the categories for certain status types are pre-defined, insome embodiments, the category for a specific type of status indicatorcan be specified via one or more user inputs. In some embodiments,device 700 displays one or more settings for a particular type of device(e.g., lights) and/or a particular type of status indicator that allowthe user to set the status category of a status indicator related to theparticular type of device and/or the particular type of status indicator(e.g., most important, second-most important, third-most urgent, etc.)

A status indicator in the fourth status indicator category indicatesthat a number of external devices of a certain type are in a certainstate. For example, as illustrated in FIG. 13B, door lock statusindicator 1314 g indicates that two door locks are in an unlocked state.As another example, in FIG. 13C, light status indicator 1314 h indicatesthat five lights are in an active (or on) state and open window statusindicator 1314 i indicates that seven window blinds (e.g., coverings,shades) are in an open state. Notably, the status indicator in thefourth status indicator category excludes information concerning devicesof the same type that are in a different state. For example, the livingroom desk lamp assigned to home 602 is off (e.g., as indicated by livingroom desk lamp control 718 g). In FIG. 13C, light status indicator 1314h does not include information concerning the living room desk lampbecause the living room desk lamp is off (as explained in further detailbelow with respect to in FIGS. 13D-13F).

FIGS. 13C-13H illustrate different gestures detected by device 700 thatare directed to a status indicator (e.g., a status indicator in thefourth status indicator category). In particular, FIGS. 13C-13Hillustrate different gestures being directed to light status indicator1314 h. However, similar techniques, as described herein, could beapplied to gestures being directed to other status indicators, such asdoor lock status indicator 1314 g and open window status indicator 1314i. At FIG. 13C, device 700 detects gesture 1350 c directed to (e.g., on,at a location corresponding to) light status indicator 1314 h.

At FIG. 13D, device 700 determines that gesture 1350 c is apress-and-hold gesture (e.g., a gesture that is detected for more than apredetermined period of time). As illustrated in FIG. 13D, becausedevice 700 has determined that gesture 1350 c is a press-and-holdgesture, device 700 displays user interface 1340 (e.g., as opposed todisplaying additional status indicator when it was determined thatgesture 1350 b was a swipe gesture). As shown in FIG. 13D, userinterface 1340 has replaced user interface 710 that was previouslydisplayed in FIG. 13C.

As illustrated in FIG. 13D, user interface 1340 includes the externaldevice controls 718 for the lights (e.g., lighting devices) that areassigned to home 602 because light status indicator 1314 h indicates astatus of one or more lights assigned to home 602. As illustrated inFIG. 13D, external device controls 718 displayed on user interface 1340are ordered by a room or a portion of home 602 (e.g., “bedroom,” “diningroom,” “living room,” “porch”) to which each respective external devicecontrol is assigned. For example, bedroom light control 718 f andbedroom night light control 718 h are assigned to the bedroom (e.g., 602c) of home 602 and are under a “BEDROOM” heading on user interface 1340,dining room light control 718 i is assigned to the dining room (e.g.,602 b) of home 602 and is under a “DINING ROOM” heading on userinterface 1340, living room lamp control 718 b and living room desk lampcontrol 718 g is assigned to the living room (e.g., 602 d) of home 602and is under a “LIVING ROOM” heading on user interface 1340, and porchlight control 718 j is assigned to the porch of home 602 and is under a“PORCH” heading on user interface 1340.

Notably, as illustrated in FIG. 13D, bedroom light control 1718 f,bedroom night light control 718 h, dining room light control 718 i,living room lamp control 718 b, and porch light control 718 j eachindicate that the respective devices (e.g., light bulbs) to which theycorrespond are on or activated. In other words, the devicescorresponding to bedroom light control 1718 f, bedroom night lightcontrol 718 h, dining room light control 718 i, living room lamp control718 b, and porch light control 718 j are on and are indicative of one ofthe “five lights on” indication as shown by light status indicator 1314h in FIG. 13C. Moreover, when looking at light status indicator 1314 h,there is no indication of a sixth light and/or the status of a sixthlight, although six lights are shown on user interface 1340. Even thoughonly five lights are on or active and identified in the light statusindicator 1314 h, the home may have more lights, e.g., lights that areinactive or off, to which the home is associated. In someimplementations, user interface 1340 shows the detailed status of alllights when displaying detailed statuses of lights. In FIG. 13D, theliving room desk lamp is displayed with a different visual indicationthan the other lights (e.g., indicating living room desk lamp control718 g is off). In some embodiments, user interface 1340 does not includeliving room desk lamp control 718 g at FIG. 13D because the living roomdesk lamp is not in the same state of the other light controls that arerepresented by light status indicator 1314 h in FIG. 13C. In someembodiments, when user interface 1340 does not include living room lampcontrol 718 g at FIG. 13D, device 700 can display living room lampcontrol 718 g on user interface 1340 in response to receiving anindication that the state of the living room desk lamp has changed to on(e.g., when someone has physically turned on the living room desk lamp)or the same state of the other light controls that are represented bylight status indicator 1314 h. At FIG. 13D, device 700 detects tapgesture 1350 d directed to bedroom light control 718 f.

At FIG. 13E, in response to detecting tap gesture 1350 d, device 700transmits an instruction that causes bedroom light 632 to change state(e.g., turn off) and updates the visual appearance of bedroom lightcontrol 718 f. As illustrated in FIG. 13E, bedroom light control 718 findicates that the bedroom light (e.g., bedroom light 632) correspondingto bedroom light control 718 f is off, e.g., in an inactive state, asopposed to being on, e.g., in an active state, as shown in FIG. 13D.Notably, the other light controls displayed on user interface 1340 havenot changed at FIGS. 13D-13E. In response to detecting tap gesture 1350d, device 700 sends an instruction that causes bedroom light 632 tochange state without causing the other lights represented by the otherlight controls to change state. Thus, user interface 1340 allows a userto cause each device represented by each respective control toindividually perform an action (e.g., change the state of the device)without causing the other devices to perform the action. At FIG. 13E,device 700 detects tap gesture 1350 e directed to done affordance 1342.

As illustrated in FIG. 13F, in response to detecting tap gesture 1350 e,device 700 ceases to display user interface 1340 and re-displays userinterface 710 that has status indicators 1314. As illustrated in FIG.13F, light status indicator 1314 h is updated to reflect that fourlights are on in home 602 as opposed to five lights being on asindicated by light status indicator 1314 h in FIG. 13C. Thus, somestatus indicators are updated to indicate the number of devices that arecurrently in a particular state (e.g., whether a device has changed fromthat particular state or whether a device has changed to that particularstate). Moreover, at FIG. 13F, bedroom light control 718 f has alsochanged on user interface 710 to reflect that the bedroom light is offwhile the other light controls (e.g., living room lamp control 718 b andliving room desk lamp control 718 g) displayed on user interface 710have not changed when compared to their respective appearances asillustrated on FIG. 13C. Therefore, the device controls displayed onuser interface 710 are also updated in response to device 700 detectinggestures (e.g., gesture 1350 d) directed to device controls displayedvia user interface 1340. At FIG. 13F, device 700 detects gesture 1350 fdirected to light status indicator 1314 h.

At FIG. 13G, device 700 determines that gesture 1350 f is a tap gesture(e.g., a gesture that is detected for less than a predetermined periodof time, a gesture that is different from gestures 1350 a-1350 c). AtFIG. 13G, because device 700 has determined that gesture 1350 f is a tapgesture, device 700 transmits an instruction that causes the four lightsindicated as on by light status indicator 1314 h in FIG. 13F to turn off(or change state). Thus, bedroom night light control 718 h, dining roomlight control 718 i, living room lamp control 718 b, and porch lightcontrol 718 j are off in FIG. 13G. As illustrated in FIG. 13G, becausedevice 700 has determined that gesture 1350 f is a tap gesture, device700 changes the visual appearance of light status indicator 1314 h(e.g., shown by the newly introduced hatching in FIG. 13G as compared tono hatching in FIG. 13F) and updates light status indicator 1314 h toindicate that the four lights have been turned off. Notably, asillustrated in FIG. 13G, living room lamp control 718 b and living roomdesk lamp control 718 g are also changed to indicate that theirrespective corresponding devices have been turned off. Thus, at FIG.13G, device 700 changes the state of multiple devices (e.g., devicesassigned to different rooms) in response to a single gesture (e.g., atap) on a status indicator, which is different from device 700 onlychanging the state of one device in response to the same type of gesture(e.g., a tap) on an external device control (e.g., as described above inrelation to gesture 1350 d in FIG. 13D). Moreover, at FIG. 13G, device700 causes the multiple devices to change state by performing the sameaction (e.g., turning lights on, locking door, close blinds) that isbased on the type of device (e.g., lights, door locks, window blindsadjusters) indicated by the status indicator to which the tap gesturewas directed. As illustrated in FIG. 13G, because gesture 1350 f in FIG.13H is a tap gesture that was directed to light status indicator 1314 h,device 700 does not change the status of the other status indicators,does not transmit an instruction that causes other devices correspondingto the other status indicators to change state, and maintains theappearance of the other status indicators (e.g., when comparing FIGS.13F-13G).

At FIG. 13H, device 700 determines that a predetermined period of timehas passed since gesture 1350 f was detected without device 700detecting another gesture directed to light status indicator 1314 h. Asillustrated in FIG. 13H, because the determination is made that thepredetermined period of time has passed since gesture 1350 f wasdetected, device 700 ceases to display light status indicator 1314 h andmoves open window status indicator 1314 i, door sensor status indicator1314 j, and motion sensor status indicator 1314 k to the left in orderto fill the space previously occupied by light status indicator 1314 h.For example, open window status indicator 1314 i was adjacent to lightstatus indicator 1314 h in FIG. 13G, and open window status indicator1314 i occupies the location vacated by light status indicator 1314 h inFIG. 13H. Because some of the status indicators have shifted to theleft, an additional status indicator, light sensor status indicator 1314k, is newly displayed. Door sensor status indicator 1314 j, motionsensor status indicator 1314 k and light sensor status indicator 1314 kare in a fifth status indicator category (e.g., a fifth-most urgentcategory, a status indicator category that is less urgent than thefourth status indicator category, a sensor status indicator category).Door sensor status indicator 1314 j indicates that a door sensorassigned to home 602 has detected that a door is ajar. Motion sensorstatus indicator 1314 k indicates that motion has been detected by oneor more motion sensors assigned to home 602. Light sensor statusindicator 13141 indicates that one or more door sensors have detectedthat one or more doors that are assigned to home 602 are opened. In someembodiments, when a gesture directed to light status indicator 1314 h isdetected before the predetermined period of time has elapsed, device 700transmits an instruction that causes the four lights to change back tobeing in the on state and updates light status indicator 1314 h toindicate that the four lights are on (e.g., as shown in FIG. 13F). Thisability to still view a status indicator for a predetermined period oftime after accessories associated with the status indicator have changedfrom an urgent or active state provides the user with the opportunity toeasily undo a state change of accessories. In some embodiments, when thegesture directed to light status indicator 1314 h is detected before thepredetermined period of time has elapsed, device 700 maintains displayof light status indicator 1314 h, irrespective of the amount of timethat passes before an additional gesture is received. In some of theseembodiments, device 700 maintains display of light status indicator 1314h because light status indicator 1314 h is indicating that one or moreof the lights are back in the original state that the one or more lightswere in when light status indicator 1314 h was first displayed on userinterface 710 (e.g., in FIG. 13B). At FIG. 13H, device 700 detects tapgesture 1350 h on room navigation control 722 b.

As illustrated in FIG. 13I, in response to detecting tap gesture 1350 h,device 700 displays user interface 1360 of the home application andindicates that room navigation control 722 b is selected (e.g., asindicated by being bold). User interface 1360 is a user interface forcontrolling multiple external devices assigned to bedroom 602 c, asindicated by bedroom identifier 1368. In some embodiments, device 700can use one or more techniques as described above in relation todisplaying user interface 760 of FIG. 7G in order to display userinterface 1360 of FIG. 13I. In some embodiments, user interface 1360 canbe alternatively displayed in response to detecting tap gesture 750 f 3in FIG. 7F and/or user interface 760 can be alternatively displayed inresponse to detecting tap gesture 1350 h in FIG. 13H.

As illustrated in FIG. 13I, user interface 1360 includes a reduced setof status indicators than the status indicators that were displayed inFIGS. 13A-13C and 13F-13H. User interface 1360 includes a reduced set ofstatus indicators because a determination is made that the respectivestatus indicators displayed on user interface 1360 correspond to atleast one device assigned to the bedroom 602. In other words, whendisplaying user interface 1360 for controlling multiple external devicesassigned to bedroom 602 c, device 700 ceases to display statusindicators that do not correspond to at least one device that isassigned to bedroom 602 c.

As illustrated in FIG. 13I, the reduced set of status indicators includecarbon monoxide status indicator 1314 a, climate status indicator 1314f, and open window status indicator 1314 i. Notably, climate statusindicator 1314 f of FIG. 13I indicates a temperature related to athermostat that is assigned to the bedroom but does not indicate thehumidity and/or air quality information that climate status indicator1314 f of FIG. 13B indicated. Here, climate status indicator 1314 f ofFIG. 13I does not indicate the humidity and/or air quality informationbecause no humidity and air quality sensors are assigned to the bedroom.However, climate status indicator 1314 f of FIG. 13I indicates thetemperature related to a thermostat control because at least onethermostat is assigned to the bedroom. Moreover, status indicator 1314 fof FIG. 13I also continues to indicate (e.g., when compared to 1314 f ofFIG. 13B) that the thermostat control is causing a heating system to beactivated (e.g., a room is being heated by a device that the thermostatcontrol is controlling), which is indicated by the upward point arrow onstatus indication 1314 f of FIG. 13I. Thus, status indicators thatcorrespond to multiple devices of the same type can cease to displayinformation concerning one or more of the types of devices whennavigating to user interfaces that correspond to different portions orrooms of home 602. Moreover, open window status indicator 1314 i of FIG.13I has changed to indicate that one window is opened as opposed toindicating that seven windows are opened, as illustrated in FIG. 13H.Open window status indicator 1314 i of FIG. 13I has changed to indicatethat one window is opened because only one bedroom window has a windowblinds adjuster device that is assigned to bedroom 602 c. At FIG. 13I,device 700 detects press-and-hold gesture 1350 i that is directed toopen window status indicator 1314 i.

As illustrated in FIG. 13J, in response to detecting press-and-holdgesture 1350 i, device 700 displays user interface 1370 that includeswindow blinds adjustable control 1318 i. At FIG. 13J, device 700 detectsgesture 1350 j (a swipe down gesture) on window blinds adjustablecontrol 1318 i. As illustrated in FIG. 13K, in response to detectinggesture 1350 j, device 700 transmits an instruction to a window blindsadjuster device assigned to bedroom 602 b that causes the windows blindsadjuster to change from being 100% (as indicated by indication 1372 ofFIG. 31J) opened to being 58% opened (e.g., as indicated by indication1372 of FIG. 13K).

Notably, at FIG. 13J, device 700 displays window blinds adjustablecontrol 1318 i, where a selection of window blinds adjustable control1318 i changes the state of a window blinds adjuster device to a statethat is different from the state (e.g., “open,” 100% open) indicated bywindow blinds adjustable control 1318 i and a state (e.g., “closed,” 0%open) that is opposite of the state (e.g., “open”) indicated by windowblinds adjustable control 1318 i. Thus, selection of window blindsadjustable control 1318 i allows a user to change the window blindsadjuster device to a state that is different from the state in which auser can change the window blinds adjuster device by directing a tapgesture toward open window status indicator 1314 i in FIG. 13I. In someembodiments, device 700 displays a user interface similar to userinterface 1340 of FIG. 13D when open window status indicator 1314 i ofFIG. 13I indicates that more than the predetermined number of devices.Thus, when there are more than a predetermined number of devicesindicated by window status indicator 1314 i of FIG. 13I, multiplecontrols are displayed that control multiple devices, and selection ofone of the respective controls causes the device to change in theopposite state (e.g., on/off state, open/closed state, locked/unlockedstate). In some embodiments, device 700 displays controls that allow auser to have more (or different) control over the one or more deviceswhen there are less than a predetermined number of devices than whenthere are more than the predetermined number of devices indicated by astatus indicator. In some embodiments, when there are less than apredetermined number of devices indicated by a status indicator and atap-and-hold gesture is detected on the status indicator, device 700displays more controls for controlling a particular device than thenumber of controls that are displayed for the particular device whenmore than the predetermined number of devices are indicated to be in aparticular state by a status indicator.

FIGS. 13L-13N illustrate exemplary user interfaces that are displayed inresponse to detecting gestures directed to other status indicators. Insome embodiments, the gesture directed to the other status indicatorscan be the same type of gesture as gesture 1350 c (e.g., press-and-holdgesture) or the same type of gesture as gesture 1350 f (e.g., a tapgesture) to display the respective user interfaces of FIGS. 13L-13N.Thus, in some embodiments, device 700 displays the respective userinterface of FIGS. 13L-13N in response to detecting either type ofgesture (e.g., a press-and-hold gesture or a tap gesture).

FIG. 13L illustrates device 700 displaying user interface 1382. Userinterface 1370 is displayed in response to device 700 detecting agesture directed to carbon monoxide status indicator 1314 a of FIG. 13A.As illustrated in FIG. 13L, user interface 1382 includes external devicecontrols 718, where each external device control indicates the status ofa carbon monoxide device assigned to home 602. As shown by userinterface 1382, the bedroom carbon monoxide sensor is trigged (e.g., asindicated by bedroom carbon monoxide control 718 k) and kitchen carbonmonoxide sensor and living room carbon monoxide sensor are not trigged(as indicated by carbon monoxides controls 718 l-718 m). Thus, inresponse to device 700 detecting a gesture directed to a statusindicator in a status indicator category other than the fourth statusindicator category (e.g., carbon monoxide status indicator 1314 a),device 700 displays the device controls of the particular type thatcorresponds to the status indicator of the respective category, wherethe device controls have one or more visual appearances that areindicative of a state that a device corresponding to a respectivecontrol is in. In some embodiments, user interface 1370 is displayedusing similar techniques to those described above in relation to FIGS.13D-13E.

FIG. 13M illustrates device 700 displaying user interface 1384. Userinterface 1384 is displayed in response to device 700 detecting agesture directed to climate status indicator 1314 f in FIG. 13B. Asillustrated in FIG. 13L, user interface 1384 includes controlscorresponding to multiple types of devices because climate statusindicator 1314 f indicates the status of multiple types of devices(e.g., as described above in relation to FIG. 13B). For example, in FIG.13L, user interface 1384 includes controls for controlling thermostatdevices (e.g., bedroom thermostat control 718 n and living roomthermostat control 718 r), controls for controlling air quality sensors(e.g., kitchen air quality sensor control 718 o and living room airquality sensor control 718 r), and a control for controlling a humiditysensor (e.g., kitchen humidity sensor control 718 p).

FIG. 13N illustrates device 700 displaying user interface 1386. Userinterface 1386 is displayed in response to device 700 detecting agesture directed to motion sensor status indicator 1314 k in FIG. 13H.As illustrated in 13N, user interface 1386 includes controlscorresponding to motion sensors that are assigned to home 602. Userinterface 1386 is displayed using similar techniques to those describedabove in relation to FIGS. 13D-13E and 13L.

FIG. 14 is a flow diagram illustrating a method for managing externaldevices using a computer system in accordance with some embodiments.Method 1400 is performed at a computer system (e.g., 100, 300, 500, 634,674, 700, 701, 850) in communication with a display generation component(e.g., a display) and one or more inputs devices (e.g., atouch-sensitive display screen). Some operations in method 1400 are,optionally, combined, the orders of some operations are, optionally,changed, and some operations are, optionally, omitted.

As described below, method 1400 provides an intuitive way for managingexternal devices. The method reduces the cognitive burden on a user formanaging external devices, thereby creating a more efficienthuman-machine interface. For battery-operated computing devices,enabling a user to recognize device status faster and more efficientlyconserves power and increases the time between battery charges.

The computer system displays (e.g., 1402), via the display generationcomponent, a first status indicator (e.g., 1314 g, 1314 h, 1314 i)(e.g., as a part of displaying a first user interface (e.g., a userinterface for controlling a set of accessory devices) that includes(e.g., includes displaying)) that indicates (e.g., a graphicalrepresentation and corresponding text (e.g., 2 light bulbs are on)) thata plurality of accessory devices of a first accessory type (e.g., lightbulbs, windows, switches, outlets, locks (e.g., door locks, windowlocks), televisions) are in a first state (e.g., on, off, open, closed,locked, unlocked). The plurality of accessory devices of the firstaccessory type includes a first accessory device (1404) that is thefirst accessory type and is in the first state and a second accessorydevice (1406) that is the first accessory type and is in the firststate, where the second accessory device is different from the firstaccessory device. In some embodiments, the first status indicator doesnot indicate that an accessory device of the first accessory type thatis not in the first state (e.g., in a second state) (e.g., the firststatus indicator does not indicate or control (e.g., when selected) oneor more devices of the first accessory type that are not in the firststate (e.g., in the second state)). In some embodiments, the firststatus indicator does not indicate an accessory device of an accessorytype that is not the first accessory type, irrespective of the currentstate of the accessory device of the accessory type that is not thefirst accessory type.

The computer system, while displaying the first status indicator, thecomputer system detects (1408) a gesture (e.g., 1350 a-1350 c, 1350 f)(e.g., via one or more input devices).

In response to (1410) detecting the gesture and in accordance with adetermination that the gesture is a first type of gesture (e.g., 1350 f)(e.g., a tap gesture) and is directed to (e.g., on, at a locationcorresponding to) the first status indicator (e.g., 1350 h), thecomputer system causes (1412) (e.g., via initiating a process (and, insome embodiments, the process includes transmitting one or moreinstructions to one or more devices (e.g., a server, a hub device thatis in communication with a plurality of devices that includes the firstaccessory device and the second accessory device, first accessorydevice, second accessory device))) the first accessory device (e.g., toperform an action that changes the state of first accessory device) andthe second accessory device (e.g., to perform an action that changes thestate of the second accessory device) to change to a second state thatis different (e.g., an opposite of (e.g., on/off, locked/unlocked,open/closed)) from the first state. In some embodiments, as a part ofcausing the first accessory device and the second accessory device to bein the second state, the computer system does not cause one or moreaccessory devices that are in the second state (or any state that isdifferent from the first state) (e.g., before the gesture was detected)to perform any actions and/or to change state.

In response to (1410) detecting the gesture and in accordance with adetermination that the gesture is a second type of gesture (e.g., 1350c) (e.g., a long press gesture) that is different from the first type ofgesture and is directed to the first status indicator (e.g., 1314 h),the computer system displays (1414), via the display generationcomponent, a first accessory control user interface object (e.g., 718 b,718 f-718 j) (e.g., as a part of displaying a second user interface)corresponding to the first accessory device, where selection (e.g., 1350d) (e.g., via a gesture that is the first type of gesture (e.g., a tapgesture)) of the first accessory control user interface object causes(e.g., via initiating a process (and, in some embodiments, the processincludes transmitting an instruction to one or more devices (e.g., aserver, a hub device that is in communication with a plurality ofaccessory devices (e.g., that includes the first and second accessorydevices)), first accessory device (not the second accessory device)) thefirst accessory device (e.g., to perform the action that changes a stateof the first accessory device) to change to the second state withoutcausing the second accessory device to change to the second state (e.g.,the second accessory device continues to be in the first state aftercausing the first accessory device to be in the first state iscompleted). In some embodiments, while displaying the first accessorycontrol user interface object corresponding to the first accessorydevice, the computer system detects a gesture (e.g., a gesture of thefirst type directed to the first accessory control user interfaceobject) directed to the first accessory control user interface object;and, in response to detecting the gesture directed to the firstaccessory control user interface object, the computer system causes thefirst accessory device (e.g., to perform the action that changes thefirst accessory device) to be in the second state without causing thesecond accessory device to be in the second state (e.g., the secondaccessory device continues to be in the first state after the causingthe first accessory device to be in the first state is completed). Insome embodiments, in response to detecting the gesture directed to thefirst status indicator and in accordance with the determination that thegesture is the second gesture (e.g., a long press gesture) that isdifferent from the first gesture, the computer system displays, via thedisplay generation component, a third accessory control user interfaceobject corresponding to a third accessory device, where the thirdaccessory device is the first accessory device type and is not in thefirst state (e.g., is in the second state); and in response to detectinga gesture directed to the third accessory control user interface object,the computer system causes the third accessory device to be in the firststate (e.g., to perform the action that changes the third accessorydevice) without changing the state of the first accessory device and thesecond accessory device (e.g., without causing the first accessorydevice and the second accessory device to be in the second state). Insome embodiments, the second user interface does not include an (e.g.,or any) accessory control user interface object corresponding to anaccessory device that is not of the first type (e.g., of a secondaccessory type that is different from the first accessory type). In someembodiments, in response to detecting the gesture directed to the firststatus indicator and in accordance with the determination that thegesture is the second gesture (e.g., a long press gesture) that isdifferent from the first gesture, the computer system does not displayaccessory control user interface objects that are of an accessory typethat is different from the first accessory type are displayed via thesecond user interface). In some embodiments, in response to detectingthe gesture directed to the first status indicator and in accordancewith the determination that the gesture is the second gesture (e.g., along press gesture) that is different from the first gesture, thecomputer system ceases to display the first status indicator. Performingdifferent operations, when a gesture is directed to the first statusindicator, based on the type of gesture detected provides the user withmore control options without cluttering the user interface. Providingadditional control of the system without cluttering the UI withadditional displayed controls enhances the operability of the system andmakes the user-system interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the system) which, additionally, reducespower usage and improves battery life of the system by enabling the userto use the system more quickly and efficiently. Causing the firstaccessory device and the second accessory device to change to a secondstate that is different from the first state provides the user with onecontrol that causes multiple devices to change state, which reduces thenumber of inputs needed to cause the multiple devices to change state.Reducing the number of inputs needed to perform an operation enhancesthe operability of the system and makes the user-system interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the system)which, additionally, reduces power usage and improves battery life ofthe system by enabling the user to use the system more quickly andefficiently.

In some embodiments, in accordance with the determination that thegesture is the second type of gesture (e.g., 1350 c) that is differentfrom the first type of gesture and is directed to the first statusindicator (e.g., 1314 h), the computer system displays, via the displaygeneration component and concurrently with the first accessory controluser interface object (e.g., 718 b, 718 f-718 j), a second accessorycontrol user interface object (e.g., 718 b, 718 f-718 j) correspondingto the second accessory device. In some embodiments, selection (e.g.,via a gesture that is the first type of gesture (e.g., a tap gesture))of the second accessory control user interface object causes (e.g., viaa process (and, in some embodiments, the process includes transmittingan instruction to one or more devices (e.g., a server, a hub device thatis in communication with a plurality of accessory devices (e.g., thatincludes the first and second accessory devices)), second accessorydevice (not the first accessory device)) the second accessory device tochange to the second state without causing the first accessory device tochange to the second state. In some embodiments, while displaying thesecond accessory control user interface object corresponding to thesecond accessory device, the computer system detects a gesture (e.g., agesture of the first type directed to the first accessory control userinterface object) directed to the first accessory control user interfaceobject; and, in response to detecting the gesture directed to the secondaccessory control user interface object, the computer system causes thesecond accessory device (e.g., to perform the action that changes thesecond accessory device) to be in the second state without causing thefirst accessory device to be in the second state (e.g., the firstaccessory device continues to be in the first state after the causingthe second accessory device to be in the first state is completed). Insome embodiments, in response to detecting the gesture directed to thesecond accessory control user interface object, the second accessorycontrol user interface object is modified from having a first visualappearance (e.g., text, size, color, pressed state (e.g., depressed,pressed)) to a second visual appearance (e.g., text, size, color,pressed state (e.g., depressed, pressed)) that is different from thefirst visual appearance. Displaying a second accessory control userinterface object corresponding to the second accessory deviceconcurrently with the first accessory control corresponding to the firstaccessory device (e.g., in accordance with the determination that thegesture is the second type of gesture that is different from the firsttype of gesture and is directed to the first status indicator) providesthe user with more control options to control individual accessorydevices without cluttering the user interface. Providing additionalcontrol of the system without cluttering the UI with additionaldisplayed controls enhances the operability of the system and makes theuser-system interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the system) which, additionally, reducespower usage and improves battery life of the system by enabling the userto use the system more quickly and efficiently.

In some embodiments, the gesture is detected while concurrentlydisplaying, with the first status indicator, a second status indicator(e.g., 1314 g, 1314 i) that indicates that a plurality of accessorydevices of a second accessory type, different from the first accessorytype, is in a third state, the plurality of accessory devices of thesecond accessory type including a third accessory device that is thesecond accessory type and is in the third state (e.g., a different statethan the first state and the second state) and a fourth accessory devicethat is the second accessory type and is in the third state. In someembodiments, the third accessory device that is the second accessorytype and is in the second state and the fourth accessory device that isthe second accessory type and is in the second state is different fromthe first accessory device and the second accessory device. In someembodiments, in accordance with a determination that the gesture is thefirst type of gesture (e.g., similar to 1350 f) (e.g., a tap gesture)and is directed to (e.g., on, at a location corresponding to) the secondstatus indicator (e.g., 1314 g, 1314 i), the computer system causes(e.g., via a process initiating a process (and, in some embodiments, theprocess includes transmitting one or more instructions to one or moredevices (e.g., a server, a hub device that is in communication with aplurality of devices that includes the first accessory device, thesecond accessory device, the third accessory device, and the fourthaccessory device))) the third accessory device and the fourth accessorydevice to change to a fourth state that is different from the thirdstate (e.g., without causing the first accessory device and the secondaccessory device to change to a different state (e.g., the second statethat is different (e.g., an opposite of (e.g., on/off, locked/unlocked,open/closed)) from the first state). In some embodiments, in accordancewith a determination that the gesture is the second type of gesture(e.g., similar to 1350 c) (e.g., a press-and-hold gesture) and isdirected to (e.g., on, at a location corresponding to) the second statusindicator (e.g., 1314 g, 1314 i), the computer system displays, via thedisplay generation component, a third accessory control user interfaceobject (e.g., as a part of displaying a second user interface)corresponding to the third accessory device. In some embodiments,selection (e.g., via a gesture that is the first type of gesture (e.g.,a tap gesture)) of the third accessory control user interface objectcauses (e.g., via a process (and, in some embodiments, the processincludes transmitting an instruction to one or more devices (e.g., aserver, a hub device that is in communication with a plurality ofaccessory devices (e.g., that includes the first, second, third, andfourth accessory devices)), third accessory device (not the fourthaccessory device)) the third accessory device (e.g., to perform theaction that changes a state of the first accessory device) to change tothe fourth state without causing the fourth accessory device to changeto the fourth state (and, in some embodiments, without displaying, viathe display generation component, the first accessory control userinterface object (e.g., as a part of displaying the second userinterface) corresponding to the first accessory device and/or the secondaccessory control user interface object) (e.g., the fourth accessorydevice continues to be in the third state after causing the thirdaccessory device has been caused to be in the fourth state). In someembodiments, in accordance with a determination that the gesture is thesecond type of gesture (e.g., a press-and-hold gesture) and is directedto (e.g., on, at a location corresponding to) the second statusindicator, the computer system displays a fourth accessory control userinterface object concurrently with the third accessory control userinterface object. In some embodiments, when the fourth accessory controluser interface object is selected, the computer system causes (e.g., viaa process (and, in some embodiments, the process includes transmittingan instruction to one or more devices (e.g., a server, a hub device thatis in communication with a plurality of accessory devices (e.g., thatincludes the first, second, third, fourth accessory devices)), fourthaccessory device (not the third accessory device)) the fourth accessorydevice to change to the fourth state without causing the third accessorydevice to change to the fourth state. Performing different operationsbased on the status indicator to which the gesture is directed and thetype of gesture detected provides the user with more control options tocontrol different accessory device types without cluttering the userinterface. Providing additional control of the system without clutteringthe UI with additional displayed controls enhances the operability ofthe system and makes the user-system interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the system) which, additionally, reducespower usage and improves battery life of the system by enabling the userto use the system more quickly and efficiently. Causing the thirdaccessory device and the fourth accessory device to change to a fourthstate provides the user with one control that causes multiple devices tochange state and reduces the number of inputs needed to cause themultiple devices to change state. Reducing the number of inputs neededto perform an operation enhances the operability of the system and makesthe user-system interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the system) which, additionally, reducespower usage and improves battery life of the system by enabling the userto use the system more quickly and efficiently.

In some embodiments, in accordance with a determination that a fifthaccessory device (e.g., device represented by 718 b in FIG. 13A) is thefirst accessory device type and the fifth accessory device (e.g., devicerepresented by 718 b in FIG. 13A) is in the first state, the pluralityof accessory devices of the first accessory type includes (e.g., asindicated by 1314 h) the fifth accessory device (e.g., that is differentfrom the first accessory device and the second accessory device). Insome embodiments, in accordance with a determination that the fifthaccessory device (e.g., device represented by 718 g in FIG. 13A) is thefirst accessory device type and the fifth accessory device (e.g., devicerepresented by 718 g in FIG. 13A) is not in the first state, theplurality of accessory devices of the first accessory type does notinclude (e.g., as indicated by 1314 h) the fifth accessory device. Insome embodiments, in accordance with a determination that the fifthaccessory device is not the first accessory device type and the fifthaccessory device is in the first state, the plurality of accessorydevices of the fifth accessory type does not include the fifth accessorydevice. In some embodiments, in accordance with a determination that thefifth accessory device is not the first accessory device type and thefifth accessory device is not in the first state, the plurality ofaccessory devices of the first accessory type does not include the fifthaccessory device. Providing a status indicator that controls accessorydevices of a first accessory device type that are in a first state anddoes not control accessory devices of the first accessory device typethat are not in the first state automatically provides the user with astatus indicator that will control accessory devices that meet certainprescribed conditions. Performing an operation when a set of conditionshas been met without requiring further user input enhances theoperability of the system and makes the user-system interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the system)which, additionally, reduces power usage and improves battery life ofthe system by enabling the user to use the system more quickly andefficiently.

In some embodiments, the first type of gesture is a tap gesture (e.g.,1350 f) (e.g., a gesture that is detected (e.g., via the one or moreinput devices) for less than a predetermined period of time). In someembodiments, the second type of gesture is a press-and-hold gesture(e.g., 1350 c) (e.g., a gesture that is detected (e.g., via the one ormore input devices) for more than a predetermined period of time).

In some embodiments, in response to detecting the gesture and inaccordance with a determination that the gesture is a third type ofgesture (e.g., 1350 a) (e.g., a swipe gesture, a gesture that moves in adirection (e.g., a lateral direction)) (e.g., and is directed to thefirst status indicator), the computer system displays (e.g., byscrolling the status indicators) a status indicator (e.g., 1314 f inFIG. 13B) that was not previously displayed (e.g., the status indicatorthat was not previously displayed indicates that a plurality ofaccessory devices of an accessory that is different from the firstaccessory device type are in the same state). In some embodiments, inaccordance with a determination that the gesture is the third type ofgesture, ceasing to display the first status indicator (e.g., byscrolling the first status indicator out of view). In some embodiments,in accordance with a determination that the gesture is a third type ofgesture, moving (e.g., in the direction (e.g., left, right, up, down,etc.) of the gesture) a plurality of displayed status indicators suchthat, after the plurality of status indicators are moved, the firststatus indicator ceases to be displayed while one or more statusindicators that were not displayed before detecting the third type ofgesture are displayed. In some embodiments, one or more controls thatare not status indicators (e.g., scene control, external accessorydevice controls) remain displayed after the gesture is detected.Performing different operations when a third type of gesture is directedto the first status indicator provides the user with more controloptions (e.g., more status indicators) without cluttering the userinterface. Providing additional control of the system without clutteringthe UI with additional displayed controls enhances the operability ofthe system and makes the user-system interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the system) which, additionally, reducespower usage and improves battery life of the system by enabling the userto use the system more quickly and efficiently.

In some embodiments, as a part of causing the first accessory device andthe second accessory device to change to the second state, the computersystem causes the first accessory device and the second accessory deviceto perform a type of action (e.g., turn off lights, unlock door locks,open blinds a certain percentage, turn off ac/heat) that is based on thefirst accessory type (e.g., lights, door locks, blinds, temperaturecontrols). Causing the first accessory device and the second accessorydevice to perform a type of action that is based on the first accessorytype allows the computer system to cause a particular action to beperformed that is based on certain prescribed conditions. Performing anoperation when a set of conditions has been met without requiringfurther user input enhances the operability of the system and makes theuser-system interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the system) which, additionally, reducespower usage and improves battery life of the system by enabling the userto use the system more quickly and efficiently.

In some embodiments, before detecting the gesture, the computer systemdisplays a third status indicator (e.g., 1314 b) (e.g., that indicatesthat a plurality of accessory devices of a second accessory type (e.g.,that is different from the first accessory type) are in a third state)adjacent to the first status indicator and a fourth status indicator(e.g., 1314 c) (e.g., that indicates that a plurality of accessorydevices of a third accessory type (e.g., that is different from thesecond accessory type and the first accessory type) are in a fourthstate) adjacent to the third status indicator (e.g., 1314 e). In someembodiments, the first status indicator, the third status indicator, andfourth status indicator are displayed in an order based on a respectivestatus indicator category (e.g., alarms (e.g., co, co2, smoke, leak,alarm), action required (e.g., accessories not responding, accounterror, low battery, updated available), significant status (things thatare important: home temperature, open doors or windows), quick actions(e.g., categories that will flip the state of accessory, doors unlocked,windows open, 5 lights on, 2 fans on), sensors (e.g., humidity level,light level “34 lux”, etc.)) that corresponds to each respective statusindicator (e.g., a respective status indicator category that eachrespective status indicator is in). In some embodiments, whiledisplaying the first status indicator, the third status indicator, andthe fourth status indicator in the order, the computer system receivesan indication that a fifth status indicator should be displayed (e.g.,the state of one or more of a different accessory device type (e.g.,than the accessory device types that are indicated by the first statusindicator, the second status indicator, and the third status indicator)has changed); and in response to receiving the indicator that theindication that the fifth status indicator should be displayed: (1) inaccordance with a determination that the fifth status indicator is in afirst status indicator category (e.g., same status indicator category asthe first status indicator), the computer system displays the fifthstatus indicator at a first location (e.g., a location that is adjacentto the first status indicator); (2) in accordance with a determinationthat the fifth status indicator is in a second status indicator category(e.g., different status indicator category from the first statusindicator category) (e.g., same status indicator category as the thirdstatus indicator), displaying the fifth status indicator at a secondlocation (e.g., a location that is adjacent to the third statusindicator) that is different from the first location; and (3) inaccordance with a determination that the fifth status indicator is in athird status indicator category (e.g., different status indicatorcategory from the first status indicator category and the third statusindicator category) (e.g., same status indicator category as the fourthstatus indicator), displaying the fifth status indicator at the thirdlocation (e.g., a location that is adjacent to the fourth statusindicator) that is different from the first location and the secondlocation). Displaying status indicators in an order based on theirrespective categories provides the user with enhanced feedbackconcerning the importance or the category of a particular statusindicator and allows a user to quickly recognize the category to which aparticular status indicator corresponds. Providing improved visualfeedback to the user enhances the operability of the computer system andmakes the user-system interface more efficient (e.g., by helping theuser to provide proper inputs and reducing user mistakes whenoperating/interacting with the computer system) which, additionally,reduces power usage and improves battery life of the computer system byenabling the user to use the computer system more quickly andefficiently.

In some embodiments, the computer system detects a change in a state ofa respective accessory device of the first accessory type. In someembodiments, in response to detecting the change in the state of therespective accessory device of the first accessory type and inaccordance with a determination that a state of a respective accessorydevice has changed to a first state, the computer system adds therespective accessory device to the plurality of accessory devices of thefirst accessory type (e.g., light bulbs, windows, switches, outlets,locks (e.g., door locks, window locks), televisions) that are in thefirst state. In some embodiments, in response to detecting the change inthe state of the respective accessory device of the first accessory typeand in accordance with a determination that the state of the respectiveaccessory device has changed to the second state (or, alternatively, haschanged to not be the first state), the computer system removes therespective accessory device from the plurality of accessory devices ofthe first accessory type (e.g., light bulbs, windows, switches, outlets,locks (e.g., door locks, window locks), televisions) that are in thefirst state. Thus, as additional accessories of a respective type aredetected to be in the respective state, the additional accessories areadded to the respective status indicator, such that the additionalaccessories are controlled by activating the status indictor; as someaccessories of a respective type are detected to no longer be in therespective state, the accessories are removed from the respective statusindicator, such that the removed accessories are not controlled byactivating the status indicator. Updating the plurality of accessorydevices based on the status of accessories allows the system to enablethe user to quickly change the status of the accessories of the sametype and state, and to individually access the accessories (e.g., via apress-and-hold). Providing additional control of the system withoutcluttering the UI with additional displayed controls enhances theoperability of the system and makes the user-system interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the system)which, additionally, reduces power usage and improves battery life ofthe system by enabling the user to use the system more quickly andefficiently

In some embodiments, the first status indicator includes an indicationof a number of accessory devices that are displayed (or updated) basedon a current number of accessory devices of the first accessory typethat are in the first state. In some embodiments, before detecting thegesture, the first status indicator includes an indication of a firstnumber of accessory devices (e.g., that are the first accessory type andare in the first state). In some embodiments, in accordance with adetermination that a tenth accessory device that is the first accessorydevice type and has changed to be in the first state, the computersystem updates the first status indicator to include an indication of asecond number of accessory devices (e.g., that are the second accessorytype and are in the second state) that is different from the indicationof the first number of accessory devices. In some embodiments, inaccordance with a determination that the second accessory has changed tonot be in the first state, the computer system updates the first statusindicator to include an indication of a third number of accessorydevices that is different from the indication of the first number ofaccessory devices and the indication of the second number of accessorydevices. In some embodiments, the second number of accessory devices isgreater than the first number of accessory devices, and the third numberof accessory devices is less than the first number of accessory devices.Displaying a status indicator that includes an indication that is basedon the current number of accessory devices allows the user to quicklyrecognize the number of devices to which the status indicated by thestatus indicator corresponds and provides the user with enhancedfeedback concerning how many devices that input on the status indicatorcould potentially impact. Providing improved visual feedback to the userenhances the operability of the computer system and makes theuser-system interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the computer system) which, additionally,reduces power usage and improves battery life of the computer system byenabling the user to use the computer system more quickly andefficiently.

In some embodiments, in accordance with a determination that the gestureis the first type of gesture (e.g., 1350 f) and is directed to the firststatus indicator (e.g., 1350 h), the computer system ceases to displaythe first status indicator (e.g., 1350 h in FIG. 13G has ceased to bedisplayed in FIG. 13H) after a predetermined period of time (e.g., 1second, 5 seconds, 30 seconds) (e.g., a predetermined period of timeafter the gesture is detected). In some embodiments, in accordance witha determination that a second gesture that is the first type of gestureis received after receiving the gesture that is the first type andbefore the predetermined time, forgoing to cease to display the firststatus indicator. In some embodiments, in accordance with adetermination the gesture is the first type of gesture and is directedto the first status indicator and in accordance with a determinationthat the status indicator that is in a first category, the computersystem ceases to display the status indicator when the accessory devicesrepresented by the status indicator have changed to a particular state(e.g., lights are off, doors are locked, alarms are not active, etc.after a second predetermined period of time) and/or after thepredetermined period of time. In some embodiments, in accordance with adetermination that the gesture is the first type of gesture and isdirected to the first status indicator and in accordance with adetermination the status indicator is in a second category that isdifferent from the first category, the computer system does not cease todisplay the status indicator when the accessory devices represented bythe status indicator have changed to a particular state (e.g., lightsare off, doors are locked, alarms are not active, etc. after a secondpredetermined period of time) and/or after the predetermined period oftime. Ceasing to display a status indicator after detecting an input ofthe first type of gesture provides the user with visual feedback that anaction has been performed to change the state of the accessory devicescontrolled via the status indicator. Providing improved visual feedbackto the user enhances the operability of the computer system and makesthe user-system interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the computer system) which, additionally,reduces power usage and improves battery life of the computer system byenabling the user to use the computer system more quickly andefficiently.

In some embodiments, the first status indicator (e.g., 1314 h) isdisplayed at a first location when the gesture is detected. In someembodiments, in accordance with a determination that the gesture is thefirst type of gesture (e.g., 1350 f) and is directed to the first statusindicator (e.g., 1314 h), the computer system ceases to display thefirst status indicator (e.g., 1314 h) at the first location and displaysa sixth status indicator (e.g., 1314 i in FIG. 13H) (e.g., that isdifferent from the first status indicator) at the first location. Insome embodiments, the sixth status indicator was adjacent to the firststatus indicator before the gesture was detected. In some embodiments,displaying the status indicator that is different from the first statusindicator at the first location includes moving, via an animation, thestatus indicator that is different from the first status indicator froma third location to the first location. Ceasing to display a statusindicator and displaying an adjacent status indicator in its place afterdetecting an input of the first type of gesture reduces the number ofinputs that are needed to delete the status indicator and display newstatus indicator (e.g., the adjacent status indicator) on a display(e.g., via the display generation component). Reducing the number ofinputs needed to perform an operation enhances the operability of thesystem and makes the user-system interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the system) which, additionally, reducespower usage and improves battery life of the system by enabling the userto use the system more quickly and efficiently.

In some embodiments, in accordance with the determination that thegesture is the first type of gesture (e.g., 1350 i) and is directed tothe first status indicator (e.g., 1314 h), the computer system visuallyupdates the first status indicator (e.g., 1314 h in FIG. 13G) such thatthe first status indicator is changed from indicating that the pluralityof accessory devices of the first accessory type is in the first state(e.g., or being displayed with a first visual appearance) to indicatethat the plurality of accessory devices of the first accessory type haschanged state (e.g., are not in the first state, are in the second state(e.g., or to being displayed with a second visual appearance that isdifferent from the first visual appearance)). In some embodiments, thefirst status indicator ceases to be displayed a predetermined period oftime after visually updating. In some embodiments, in accordance withthe determination that the gesture is the second type of gesture (or thethird type of gesture) and is directed to the first status indicator,the computer system forgoes to update the first status indicator toindicate that the plurality of accessory devices of the first accessorytype is in the second state (and/or maintaining the first statusindicator (e.g., to) indicate that the plurality of accessory devices ofthe first accessory type are in the first state). Updating a statusindicator to indicate that the plurality of accessory devices havechanged state after detecting an input of the first type of gestureprovides the user with visual feedback that an action has been performedto change the state of the accessory devices controlled via the statusindicator. Providing improved visual feedback to the user enhances theoperability of the computer system and makes the user-system interfacemore efficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the computersystem) which, additionally, reduces power usage and improves batterylife of the computer system by enabling the user to use the computersystem more quickly and efficiently.

In some embodiments, the first status indicator (e.g., 1314 h) isdisplayed on a first user interface (e.g., 710). In some embodiments,the computer system receives (e.g., while displaying the first statusindicator on the first user interface) a request (e.g., 1350 h) todisplay a second user interface (e.g., 1360) that is different from thefirst user interface (e.g., 710). In some embodiments, in response toreceiving the request (e.g., 1350 h) to display the second userinterface, the computer system displays the second user interface (e.g.,1360) that includes a status indicator (e.g., 1350 i in FIG. 13I) thatindicates that one or more accessory devices of the first accessory typethat are in the first state, where: in accordance with a determinationthat the second user interface (e.g., 1360) (e.g., user interface thatis currently displayed) corresponds to a first group of accessorydevices (e.g., a group of a devices that are assigned to a grouprepresenting a portion (e.g., room (e.g., bedroom, living room, office,garage), porch) of a building (e.g., a home))) and the first accessorydevice corresponds to the first group of accessory devices (e.g., andwhile the first accessory device is in the first state), the one or moreaccessory devices of the first accessory type includes the firstaccessory device; and in accordance with a determination that the seconduser interface (e.g., 1360) (e.g., user interface that is currentlydisplayed) corresponds to the first group of accessory devices and thefirst accessory device does not correspond to the first group ofaccessory devices (e.g., and while the first accessory device is in thefirst state), the one or more accessory devices of the first accessorytype does not include the first accessory device. In some embodiments,in response to receiving the request to display the second userinterface, the computer system ceases to display the first userinterface that includes the first user interface. Displaying a statusindicator that indicates a state of a plurality of devices when certainprescribed conditions are met (e.g., based on whether the user interfacethat includes the status corresponds to a group of accessory devices andwhether a particular device that is relevant (e.g., same state andaccessory type) to the status indicator corresponds to the group ofaccessory devices) automatically provides the user with a statusindicator that indicates a plurality of devices that are relevant to theuser interface that corresponds to the particular group of accessorydevices. Performing an operation when a set of conditions has been metwithout requiring further user input enhances the operability of thesystem and makes the user-system interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the system) which, additionally, reducespower usage and improves battery life of the system by enabling the userto use the system more quickly and efficiently. Displaying a statusindicator that a state of a plurality of devices based on whether theuser interface that includes the status corresponds to a group ofaccessory devices and whether a particular device that is relevant(e.g., same state and accessory type) to the status indicatorcorresponds to the group of accessory devices provides the user withvisual feedback concerning the status indicators that indicate devicesthat are relevant to the particular user interfaces being displayed.Providing improved visual feedback to the user enhances the operabilityof the computer system and makes the user-system interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the computersystem) which, additionally, reduces power usage and improves batterylife of the computer system by enabling the user to use the computersystem more quickly and efficiently.

In some embodiments, before detecting the gesture (e.g., 1350 a-1350 cand 1350 f), an eighth status indicator that indicates that one or moreaccessory devices of a third accessory type are in a seventh state(e.g., on/off, open, closed) is displayed. In some embodiments, the oneor more accessory devices include an eighth accessory device of thethird accessory type. In some embodiments, in accordance with adetermination that the gesture (e.g., 1350 f, 1350 i) is the second typeof gesture (e.g., a press-and-hold gesture) and is directed to (e.g.,on, at a location corresponding to) the eighth status indicator (e.g.,1314 h) and in accordance with a determination that the eighth statusindicator (e.g., 1314 h in FIG. 13I) corresponds to (and/or indicatesthat) less than a predetermined number (e.g., less than 2) of accessorydevices of the third accessory type that are in the seventh state (e.g.,only 1 accessory of the third accessory type is in the seventh state),the computer system displays an eighth accessory control user interfaceobject (e.g., 1318) (e.g., with more control and/or precision than theninth accessory control user interface object) corresponding to theeighth accessory device. In some embodiments, selection of the eighthaccessory control user interface object causes the eighth accessorydevice to change to an eighth state (e.g., 80% brightness, 20%brightness, 40% opened, 20% opened, a state that is not completelyopposite of the seventh state, state that corresponds to a differentcharacteristic (e.g., color (e.g., blue, red, green, yellow, white) oflights) that can be represented by the device than the characteristic(e.g., on/off) that corresponds to the seventh state). In someembodiments, in accordance with a determination that the gesture (e.g.,1350 f, 1350 i) is the second type of gesture (e.g., a press-and-holdgesture) and is directed to (e.g., on, at a location corresponding to)the eighth status indicator (e.g., 1314 h) and in accordance with adetermination that the eighth status indicator (e.g., 1314 h in FIG.13I) corresponds to (e.g., and/or indicates that) not less than thepredetermined number (e.g., 2 or more) of the accessory devices of thethird accessory type are in the seventh state, the computer systemdisplays a ninth accessory control user interface object (e.g., controllike, for example, 718 h) corresponding to the eighth accessory device.In some embodiments, selection of the ninth accessory control userinterface object causes the eighth accessory device to change to a ninthstate (e.g., on/off, open/closed, an opposite state than the seventhstate) that is different from the eighth state (e.g., without displayingthe eighth accessory control user interface object). In someembodiments, the number of accessory user interface control objects thatcorrespond to the eighth accessory device is greater when thedetermination is made that the eighth status indicator indicates thatless than the predetermined number (e.g., 2) of accessory devices of thethird accessory type are in the seventh state than when thedetermination is made that the eighth status indicator indicates thatless than the predetermined number (e.g., 2) of the third accessory typeis in the seventh state. In some embodiments, the eighth accessorycontrol user interface object is larger than the ninth accessory controluser interface object. Displaying controls that change, when selected,an accessory device differently based on the number of controlsrepresented by the status indicator allows the device to provideadditional or different controls in some situations without clutteringthe UI. Providing additional control of the system without clutteringthe UI with additional displayed controls enhances the operability ofthe system and makes the user-system interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the system) which, additionally, reducespower usage and improves battery life of the system by enabling the userto use the system more quickly and efficiently.

In some embodiments, in accordance with a respective status indicator(e.g., the eighth status indicator) corresponding to a single accessorydevice, the respective status indicator includes a name (e.g.,alphanumeric text) of the accessory device. In accordance with therespective status indicator (e.g., the eighth status indicator)corresponding to a plurality of accessory devices, the respective statusindicator includes an indication (e.g., a numeric representation) of thenumber of accessory devices represented.

In some embodiments, the computer system detects a tap on another typeof status indicator (e.g., 1314 a-1314 f and 1314 j-1314 l) (e.g.,climate control, sensors (e.g., different types of sensors may becombined in a single status indicator)) and, in response to detectingthe tap, displays a user interface (e.g., 1382, 1384, 1386) forcontrolling the plurality of a type of devices related to the other typeof status indicator. In some embodiments, device 700 does not ceases todisplay the other type of control after detecting the tap on it.

Note that details of the processes described above with respect tomethod 1400 (e.g., FIG. 14) are also applicable in an analogous mannerto the methods described above. For example, method 1400 optionallyincludes one or more of the characteristics of the various methodsdescribed above with reference to methods 800 and 1000. For anotherexample, the audio message sent in method 800 can be change or morestatus indicators as described above in relation to method 1400. Forbrevity, these details are not repeated below.

FIGS. 15A-15F illustrate exemplary user interfaces for playing backaudio messages in accordance with some embodiments. The user interfacesin these figures are used to illustrate the processes described below,including the processes in FIG. 16.

FIG. 15A illustrates device 700 displaying user interface 702 thatincludes affordances 704. As illustrated in FIG. 15A, multicastnotification 1502 is displayed over a portion of user interface 702.Multicast notification 1502 indicates that audio multicast messages havebeen sent out from John and three others to a group of devices assignedto home 602, which includes device 700. In other words, because device700 is assigned to home 602, device 700 has received the audio multicastmessages from John and three others. Multicast notification 1502includes play representation 1504. In some embodiments, playrepresentation 1504 is selectable. In some embodiments, multicastnotification 1502 can include one or more features, as described abovein relation to the other multicast messages described above (e.g.,multicast notification 708 a, multicast notification 708, multicastnotification 904, multicast notification 906 a, multicast notification906 b, multicast notification 908 a, multicast notification 908 b,multicast notification 908 c, multicast notification 910). In someembodiments, the other multicast messages described above can includeone or more features (e.g., play representation 1504) of multicastnotification 1502. At FIG. 15A, device 700 detects tap gesture 1550 a onmulticast notification 1502.

FIGS. 15B-15F illustrate exemplary user interfaces for playing backaudio multicast messages (e.g., automatically). In some embodiments, theuser interfaces described in relation to FIGS. 15B-15F can be displayedin lieu of one or more user interfaces described above, such as userinterfaces of 9I-9J and/or user interfaces of 9M-9P.

As illustrated in FIG. 15B, in response to detecting tap gesture 1550 a,device 700 displays user interface 1520 overlaid on to a portion of userinterface 702 and automatically (e.g., without intervening user inputafter detecting tap gesture 1550 a) initiates playback of an audiomulticast message received from John, as indicated by current senderindication 1322 a. In some embodiments, as a part of initiating playbackof audio, device 700 outputs audio via one or more speakers that areelectronically coupled to device 700.

As illustrated in FIG. 15B, user interface 1520 includes multicast homeidentifier 1532, which indicates that audio multicast messages that weresent to a group of devices associated with home 602 are being playedback. User interface 1520 also includes multicast recording affordance736, current sender indication 1522 a, graphical sender indications1524, and count indication 930. In some embodiments, device 700 uses oneor more techniques described above to display and operate in view ofmulticast recording affordance 736, current sender indication 1522 a,graphical sender indications 1524, and count indication 930 that device700 uses to, respectively, display and operate in view of multicastrecording affordance 736, current sender indication 922 a-922 b,graphical sender indications 924 a-924 b, and count indication 930described above.

Looking back at FIG. 15A, notification 1502 indicated that four audiomulticast messages were received (e.g., “John and 3 others”) to beplayed back. Accordingly, in FIG. 15B, user interface 1520 includesgraphical sender indications 1524 corresponding to a portion of thereceived messages. In FIG. 15B, graphical sender indications 1524include graphical sender indications 1524 a-1524 c. Graphical indication1524 a corresponds to a first audio multicast message, which wasreceived from “John.” Graphical indication 1524 b corresponds to asecond audio multicast message that was received (or sent) after thefirst audio multicast message. Graphical indication 1524 c correspondsto a third audio multicast message that was received (or sent) after thesecond audio multicast message. Notably, in FIG. 15A, user interface1520 does not include a graphical indication that corresponds to thefourth audio multicast message because a determination has been madethat no more than a predetermined number (e.g., three) graphicalindications should be concurrently displayed.

Graphical indications 1524 a-1524 c correspond to different multicastmessages that were sent by different users (or devices), which areillustrated by graphical indications 1524 a-1524 c depicting differentrepresentations. Graphical indication 1524 a includes a visualrepresentation of a sender (e.g., “John”) (e.g., a determined sender) ofthe first audio multicast message. Graphical indication 1524 b includesa visual representation of a device (e.g., a speaker) that recorded thesecond audio multicast message. Graphical indication 1524 c includes avisual representation of text that corresponds to a sender (e.g., “EA,”which stands for “Emma Appleseed,” who is a person associated with home602 as indicated in FIG. 7AB) of the third audio multicast message.Graphical indication 1524 b includes a visual representation of a deviceinstead of a visual representation of a sender because a person whoproduced the second audio multicast message could not be determined.Graphical indication 1524 c includes a visual representation of textinstead of a visual representation of a user because device 700 does nothave access to a visual representation of a user (e.g., Emily). In someembodiments, multiple graphical indications include the same visualrepresentations when a determination is made that the audio multicastmessages corresponding to the graphical indications were sent by thesame sender.

As illustrated in FIG. 15B, graphical indication 1524 a is the largestgraphical indication displayed on user interface 1520 and is displayedwith audio indication 1524 a 1 (e.g., circles surrounding graphicalindication 1524 a). As indicated by current sender indication 1522 a,the first audio multicast message from John is currently being playedback. Accordingly, graphical indication 1524 a is the largest graphicalrepresentation and is displayed with audio indication 1524 a 1 toindicate that the audio multicast message from John is currently beingplayed back. In some embodiments, the audio indication 1524 a 1 isdynamic and varies in size in accordance with the first audio multicastmessage being played back. Graphical indication 1524 b is the secondlargest graphical indication displayed on user interface 1520 becausethe second audio multicast message is the next audio multicast messagein the queue to be played back by device 700. Graphical indication 1524c is the third largest graphical indication displayed on user interface1520 because the third audio multicast message is the next audiomulticast message in the queue to be played back by device 700 after thesecond audio multicast message. At FIG. 15B, device 700 determines thatthe first audio multicast message will end within a predetermined periodof time (e.g., 0 second, 1 second) (or the next message (e.g., thesecond message) will begin within a predetermined period of time).

In response to determining that the first audio multicast message willend within a predetermined period of time (or has ended) (e.g., withoutuser input, based on a tap gesture on skip affordance 1534), device 700rotates graphical indications 1524 b-1524 c counter-clockwise and ceasesto display graphical indication 1524 a. As illustrated in FIG. 15C, inresponse to determining that the first audio multicast message will endwithin a predetermined period of time (or has ended), graphicalindication 1524 b is displayed at a position at which graphicalrepresentation 1524 a was previously displayed in FIG. 15B, andgraphical indication 1524 c is displayed at a position at whichgraphical representation 1524 b was previously displayed in FIG. 15B. Asillustrated in FIG. 15C, graphical indication 1524 a is displayed at thesame size at which graphical indication 1524 b was previously displayedin FIG. 15B, and graphical indication 1524 c is displayed at the samesize at which graphical indication 1524 b was previously displayed inFIG. 15B.

As shown in FIG. 15C, current sender indication 1522 b has replacedcurrent sender indication 1522 a, which was previously displayed in FIG.15B. Thus, at FIG. 15C, device 700 is currently playing back the secondaudio multicast message because playback of the first audio multicastmessage has completed by the time that user interface 1520 of FIG. 15Cis displayed. As shown in FIG. 15C, graphical indication 1524 b is nowdisplayed with audio indication 1524 b 1, which is displayed usingsimilar techniques to those described above in relation to audioindication 1524 a 1. As illustrated in FIG. 15C, graphical indication1524 c does not have an audio indication surrounding graphicalindication 1524 c because device 700 is not currently outputting thethird audio multicast message (e.g., an audio multicast message thatcorresponds to graphical indication 1524 c). In some embodiments, device600 displays an animation gradually increasing the sizes of graphicalindications 1524 b-1524 c while transitioning between displaying userinterface 1520 of 15B and displaying user interface 1520 of FIG. 15C.

As illustrated in FIG. 15C, because device 700 has ceased to displaygraphical indication 1524 a (e.g., the first audio multicast message hasended), device 700 newly displays graphical indication 1524 d, whichcorresponds to the fourth audio multicast message that was sent by Jane(as shown by current sender indication 1522 d in FIG. 15E). Graphicalindication 1524 d is displayed at the same size of and in the samelocation as graphical indication 1524 c of FIG. 15B (e.g., becausegraphical indication 1524 d is now second in the queue for audioplayback). At FIG. 15C, device 700 determines that the second audiomulticast message will end within a predetermined period of time (e.g.,0-1 second) (or the next message (e.g., the third audio multicastmessage) will begin within a predetermined period of time) (or hasended).

In response to determining that the second audio multicast message willend within a predetermined period of time (or has ended), device 700rotates graphical indications 1524 c-1524 d counter-clockwise and ceasesto display graphical indication 1524 b. As illustrated in FIG. 15D, inresponse to determining that the second audio multicast message will endwithin a predetermined period of time, graphical indication 1524 c isdisplayed at a position and size at which graphical representation 1524b was previously displayed in FIG. 15C, and graphical indication 1524 dis displayed at a position and a size at which graphical representation1524 c was previously displayed in FIG. 15C.

As shown in FIG. 15D, current sender indication 1522 c has replacedcurrent send indication 1522 b, which was previously displayed in FIG.15C. Thus, at FIG. 15D, device 700 is currently playing back the thirdaudio multicast message because playback of the second audio multicastmessage has completed by the time that user interface 1520 of FIG. 15Dis displayed. Device 700 plays back the third audio multicast messageautomatically, without intervening input, at a predetermined period oftime after playback of the second audio multicast message ends. In someembodiments, device 700 plays a predetermined sound (e.g., a chime, abuzz, a tone) between multicast messages that are played back. Forexample, device 700 can play a first sound before automatically startingplayback of the second audio multicast message and can play a secondsound (e.g., the same chime) before automatically starting playback ofthe third audio multicast message. In some embodiments, device 600transitions the user interfaces (e.g., user interface 1520 of FIG. 15Ctransitioning to the user interface 1520 of FIG. 15D) while (after orbefore) playing back the predetermined sound. In some embodiments, inresponse to detecting an input on skip affordance 1534 of FIG. 15C,device 700 ends playback of the second audio message (e.g., being playedback in FIG. 15C) (before reaching the end of the second audio message)and starts playback of the third audio message and transitions the userinterfaces (e.g., user interface 1520 of FIG. 15C transitioning to theuser interface 1520 of FIG. 15D). At FIG. 15D, device 700 determinesthat the third audio multicast message will end within a predeterminedperiod of time (e.g., 0-1 second) (or the next message (e.g., the fourthaudio multicast message) will begin within a predetermined period oftime) (or has ended).

In response to determining that the third audio multicast message willend within a predetermined period of time (or has ended), device 700moves (e.g., changes the position of) and enlarges graphical indication1524 d counter-clockwise and ceases to display graphical indication 1524c. As illustrated in FIG. 15E, because graphical indication 1524 d isthe only displayed graphical indication, device 700 increase size ofgraphical indication 1524 d to a size that is greater than the size atwhich graphical indication 1524 c was previously displayed. In someembodiments, graphical indication 1524 d is displayed at a differentlocation in FIG. 15E (e.g., at a center of 1524) as compared to thelocation of graphical indication 1524 a in FIG. 15B. As shown in FIG.15E, current sender indication 1522 d has replaced current sendindication 1522 c, which was previously displayed in FIG. 15D. Thus, atFIG. 15E, device 700 is currently playing back the fourth audiomulticast message (e.g., the audio multicast message that corresponds tographical indication 1524 d) because playback of the third audiomulticast message has completed by the time that user interface 1520 ofFIG. 15E is displayed. Device 700 plays back the fourth audio multicastmessage automatically, without intervening input, at a predeterminedperiod of time after playback of the third audio multicast message ends(e.g., once the third audio multicase message has ended).

As illustrated in FIG. 15F, after determining that playback of thefourth audio message has ended, device 700 updates user interface 1520to include graphical indications 1524 a-1524 c without playing backadditional media (or multicast audio messages). In addition tore-displaying graphical indication 1524 a-1524 c, device 700 displaysplayback affordance 926. In some embodiments, in response to detectingtap gesture 1550 f 1 on playback affordance 926, device 700 initiatesplayback of the first audio multicast message and repeats the flowdescribed above in FIGS. 15B-15E. In some embodiments, in response todetecting tap gesture 1550 f 2 (or a press-and-hold gesture) onmulticast affordance multicast recording affordance 736, device 700displays a user interface for recording and send an audio message to thegroup of devices associated with home 602 (e.g., to reply to the groupof devices to which the multicast audio messages (e.g., first, second,third, fourth audio multicast messages) were sent). Thus, in someembodiments, device 700 displays user interface 741 of FIG. 7C1, userinterface 730 of FIG. 7C, or user interface 740 of FIG. 7D in responseto detecting a tap gesture on multicast recording affordance 736. Insome embodiments, a tap gesture on multicast recording affordance 736starts audio recording until an end recording input (e.g., another tapgesture) is detected. Once the end recording input is detected, therecorded audio message is sent to the group of devices. In someembodiments, a press-and-hold gesture is detected on multicast recordingaffordance 736. When the press-and-hold gesture is detected, audiorecording begins. Once an end (e.g., lift-off) of the press-and-holdgesture is detected, the audio recording ends, and the recorded audiomessage is sent to the group of devices. In some embodiments, detectingactivation (e.g., tap gesture, tap-and-hold gesture) of multicastrecording affordance 736 in any of the user interfaces in FIGS. 15B-15Ecauses playback of the currently playback message to stop and startingthe process for recording an audio message to send to the group ofdevices.

FIG. 16 is a flow diagram illustrating a method for playing back audiomessages using a computer system in accordance with some embodiments.Method 1600 is performed at a computer system (e.g., 100, 300, 500, 634,674, 700, 701, 850) that is in communication with a display generationcomponent (e.g., an integrated display, a connected display) and one ormore inputs devices (e.g., a touch-sensitive surface). Some operationsin method 1600 are, optionally, combined, the orders of some operationsare, optionally, changed, and some operations are, optionally, omitted.

As described below, method 1600 provides an intuitive way for playingmedia content. The method reduces the cognitive burden on a user forplaying media messages, thereby creating a more efficient human-machineinterface. For battery-operated computing devices, enabling a user toplay media content faster and more efficiently conserves power andincreases the time between battery charges.

The computer system (e.g., 700) displays (1602), via the displaygeneration component, a media user interface (e.g., 1520) that includes:at a first location, a first representation (1604) (e.g., 1524 a in FIG.15B) of a first media message (e.g., audio media; visual media;audiovisual media), and at a second location different from the firstlocation, a second representation (1606) (e.g., 1524 b in FIG. 15B) of asecond media message. In some embodiments, the media user interfaceincludes representations of a plurality of (e.g., some of, not all of)media messages of a group of media (e.g., the group of media being in anordered set).

In response to displaying the media user interface and not based on userinput corresponding to the media user interface (e.g., 1520), thecomputer system (e.g., 700) plays (1608) the first media message (e.g.,outputting audio and/or video of the first media message).

In response to detecting completion of playing through the first mediamessage (e.g., reaching an end of the first media message by playingthrough to the end of the first media message), the computer system(e.g., 700) displays (1610), the second representation (e.g., 1524 b) ofthe second media message (e.g., as in FIG. 15C) without displaying, viathe display generation component, a representation (e.g., 1524 a) (e.g.,any representation) corresponding to the first media message (e.g., asin FIG. 15C).

Displaying representations of various media message at differentlocations provides the user with feedback about the order of the mediamessage and, for example, which media message is currently playing andwhich media message will play next. Providing improved visual feedbackto the user enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some embodiments, the representations of the plurality of (e.g., someof, not all of, three of) media message are displayed with varying sizesbased on the order of the ordered set. In some embodiments, the earlierthe media message is in the ordered set the larger the representation ofthe corresponding media message (e.g., a representation of a currentlyplaying media message is the largest, a representation of a mediamessage to be played next is the next largest, etc). Thus, the order ofthe media message is reflected in the comparative sizes of theircorresponding displayed representations.

In accordance with some embodiments, displaying, the firstrepresentation (e.g., 1524 a in FIG. 15B) of the first media message(e.g., audio media; visual media; audiovisual media) at the firstlocation includes displaying the first representation (e.g., 1524 a) ofthe first media message (e.g., audio media; visual media; audiovisualmedia) at the first location and at a first size (e.g., as in FIG. 15B).In accordance with some embodiments, displaying the secondrepresentation (e.g., 1524 b in FIG. 15B) of the second media message atthe second location includes displaying the second representation (e.g.,1524 b) of the second media message at the second location and at asecond size different from (e.g., smaller than) the first size (e.g., asin FIG. 15B).

Displaying representations of various media message at different sizesprovides the user with feedback about the order of the media message(e.g., bigger size is earlier in the ordered set) and, for example,which media message is currently playing and which media message willplay next. Providing improved visual feedback to the user enhances theoperability of the device and makes the user-device interface moreefficient (e.g., by helping the user to provide proper inputs andreducing user mistakes when operating/interacting with the device)which, additionally, reduces power usage and improves battery life ofthe device by enabling the user to use the device more quickly andefficiently.

In accordance with some embodiments, the computer system (e.g., 700)displays, via the display generation component, concurrently with thefirst representation of the first media message at the first locationand with the second representation of the second media message at thesecond location: at a third size (e.g., the same as the second size,smaller than the second size) different from the first size (and,optionally, different from the second size) and at a third locationdifferent from the first location and second location, a thirdrepresentation (e.g., 1524 c) of a third media message. In response todetecting completion of playing through the first media message (e.g.,reaching an end of the first media message by playing through to the endof the first media message): the computer system (e.g., 700) displays,at the first size and at the first location, the second representation(e.g., 1524 b) of the second media message (e.g., as in FIG. 15C)without displaying, via the display generation component, the firstrepresentation (e.g., 1524 a) of the first media message; and thecomputer system (e.g., 700) displays, at the second size and at thesecond location, the third representation (e.g., 1524 c) of the thirdmedia message (e.g., as in FIG. 15C).

Displaying representations of various media message at differentlocations and at different sizes provides the user with feedback aboutthe order of the media message to be played. Providing improved visualfeedback to the user enhances the operability of the device and makesthe user-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In accordance with some embodiments, the computer system (e.g., 700)receives (e.g., via a wireless network) data (e.g., first, second,third, and fourth data) corresponding to media message (e.g., the groupof media, the first, second, third, and fourth media message) includingthe first media message that is a first (e.g., audio) message (e.g.,directed to a target group of devices (e.g., a plurality of devices of ahome (e.g., a home automation system)), including the computer system)from a first remote user and the second media message that is a second(e.g., audio) message (e.g., directed to the target group of devices(e.g., the plurality of devices of the home (e.g., a home automationsystem)), including the computer system) from a second remote user thatis different from the first remote user. The computer system (e.g., 700)displays (e.g., in response to receiving the data, until an end of atime period (as described in further detail above)), via the displaygeneration component, a notification (e.g., 1502) corresponding to themedia message. While displaying the notification (e.g., 1502), thecomputer system (e.g., 700) receives, via the one or more input devices,selection (e.g., 1550 a) of the notification (e.g., activation of thenotification displayed on a lock screen, the notification correspondingto the group of media, the group of media being in an ordered set). Insome embodiments, displaying the media user interface occurs in responseto receiving selection (e.g., 1550 a) of the notification (e.g., 1502).In some embodiments, playing the first media message occurs in responseto receiving selection (e.g., 1550 a) of the notification (e.g., 1502).

Displaying the media user interface and automatically playing the firstmedia message in response to a selection of a notification enables thecomputer system to perform two functions (display and playback) inresponse to the same input, thereby reducing the need for the computersystem to receive additional inputs to start the playback. Reducing thenumber of inputs needed to perform an operation enhances the operabilityof the device and makes the user-device interface more efficient (e.g.,by helping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some embodiments, the first representation (e.g., 1524 a), the secondrepresentation (e.g., 1524 b), and the third representation (e.g., 1524c) are not displayed in a line (e.g., the first, second, and thirdlocations are not all on a line). In some embodiments, therepresentations of media that are displayed (e.g., as representationsremoved and new representations are add) are not displayed in a line.

In accordance with some embodiments, the first size is larger than thesecond size. In some embodiments, the second size is larger than thethird size. In some embodiments, the second size and the third size arethe same size. In some embodiments, the representation (e.g., 1524 a inFIG. 15B, 1524 b in FIG. 15C) of the currently playing media file isdisplayed at a size that is larger than the size of multiple other(e.g., all) displayed representations of media message.

Displaying representations of various media message at increasinglylarger sizes provides the user with feedback about the order of themedia message. For example, the larger the representation the earlierthe corresponding media message is in the ordered set of media message,with (for example) the largest representation being the currentlyplaying media item. Providing improved visual feedback to the userenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

In accordance with some embodiments, in response to detecting completionof playing through the first media message (e.g., reaching an end of thefirst media message by playing through to the end of the first mediamessage), playing the second media message (e.g., outputting audioand/or video of the second media) without receiving (or requiring)further user input. In some embodiments, the second media message isplayed in conjunction with displaying, at the first size and at thefirst location, the second representation of the second media message.

Automatically playing the second media message in response completion ofthe first media message eliminates the need for the computer system toreceive additional inputs to start the playback of the second mediamessage. Reducing the number of inputs needed to perform an operationenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

In some embodiments, in response to detecting completion of playing thesecond media message (e.g., reaching an end of the second media message;skipping the remainder of the first media), the computer system playsthe third media message (e.g., outputting audio and/or video of thethird media message) without receiving (or requiring) further userinput. In some embodiments, the third media message is played inconjunction with displaying, at the first size and at the firstlocation, the third representation of the third media message. In someembodiments, a fifth representation of a fifth media is not displayedafter detecting completion of playing the second media message.

In accordance with some embodiments, prior to detecting completion ofplaying through the first media message, a fourth representation (e.g.,1524 d) of a fourth media message (e.g., audio media; visual media;audiovisual media) is not concurrently displayed with the firstrepresentation of the first media (e.g., in the media user interface, asshown in FIG. 15B). After (e.g., in response to) detecting completion ofplaying through the first media message (e.g., reaching an end of thefirst media message by playing through to the end of the first mediamessage) (In some embodiments, in response to detecting completion ofplaying the first media message), the computer system (e.g., 700)displays, via the display generation component, (e.g., at the third sizeand at the third location) the fourth representation (e.g., 1524 d) ofthe fourth media message (e.g., in the media user interface (e.g.,1520)) (In some embodiments, the fourth media message is displayedconcurrently with the second and third representations of mediamessage).

Displaying representations of a first predetermined number of mediamessage of the group of media message without displaying representationsof additional media message in the group of media until media message isplayed and the representation of the played media message is removedprovides the user with feedback that additional media message areavailable for playback without cluttering the user interface. Providingimproved visual feedback to the user enhances the operability of thedevice and makes the user-device interface more efficient (e.g., byhelping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some embodiments, when the media user interface is initiallydisplayed (e.g., in response to selection of a displayed notification),the media user interface includes the first representation, the secondrepresentation, and the third representation without including thefourth representation.

In some embodiments, in response to detecting completion of playing thethird media message (e.g., reaching an end of the third media message;skipping the remainder of the third media message), displaying, at asize different from (e.g., larger than) the first size and at a locationdifferent from the first location, the fourth representation (e.g., 1524d) of the fourth media message and playing the fourth media messagewithout receiving (or requiring) user input (e.g., as shown in FIG.15E).

In accordance with some embodiments, while playing the second mediamessage, the computer system (e.g., 700) displays a next media userinterface object (e.g., 1534) in the media user interface (e.g., 1520)that, when activated (e.g., detecting a tap on), causes completion ofplaying of the second media message (e.g., ceasing playing the currentlyplaying media message; forgoing playing the remainder of the secondmedia message; skipping past the remainder of the second media message)(and, optionally, begins playing the third media message). While playingthe second media message, the computer system (e.g., 700) receives, viathe one or more input devices, activation of (e.g., a tap on) the nextmedia user interface object (e.g., 1534). In response to receivingactivation of the next media user interface object (e.g., 1534), thecomputer system (e.g., 700) displays, via the display generationcomponent, the third representation of the third media message at thefirst location (and, optionally, at the first size) without displaying,via the display generation component, the second representation of thesecond media message. In some embodiments, the third media message playsin conjunction with displaying the third representation of the thirdmedia message at the first location at the first size. In someembodiments, the next media user interface object, when activated,causes completion of the currently playing media message. In someembodiments, detecting completion of the currently playing media messagecauses (e.g., except when the currently playing media message is thefinal media message of the group of media) the technique to proceed toplaying the next media message in the ordered set. In some embodiments,in response to receiving activation of the next media user interfaceobject (e.g., 1534), the computer system ceases playing the second mediamessage (e.g., without playing through to the end of the second mediamessage) and plays the third media message.

Displaying a next media user interface object enables the computersystem to receive user input to skip to playing the next media messagewithout spending additional time playing the remainder of the currentlyplaying media message. Providing the user with this additional controlenhances the operability of the device and makes the user-deviceinterface more efficient (e.g., by helping the user to provide properinputs and reducing user mistakes when operating/interacting with thedevice) which, additionally, reduces power usage and improves batterylife of the device by enabling the user to use the device more quicklyand efficiently.

In some embodiments, the computer system does not display the previousmedia user interface object (e.g., concurrently with playing the secondmedia) (e.g., in the media user interface) that, when activated (e.g.,detecting a tap on), causes completion of playing the (e.g., second)media message and begins playing a previous (e.g., first) media message.In some embodiments, the computer system forgoes displaying the nextmedia user interface object when a last (e.g., the fourth) media message(e.g., of the group of media) is playing (e.g., as in FIG. 15E).

In some embodiments, the first media message, the second media message,and the third media message are part of a group of media. In accordancewith some embodiments, subsequent to (e.g., in response to detection ofcompletion of) playing a final media message (e.g., after playing orskipping through all media message of the group of media), the computersystem (e.g., 700) displays, via the display generation component, areplay user interface object (e.g., 926, at the first location, at thelocation at which the fourth representation of the fourth media messagewas displayed) in the media user interface that, when activated (e.g.,1550 f 1, detecting a tap on), plays the first media message (e.g.,outputting audio and/or video of the first media message) beforeproceeding to playing subsequent media message (e.g., the second, third,and fourth media message). In some embodiments, activation of the replayuser interface object restarts the process again to replay the mediamessage of the group of media. In some embodiments, subsequent toplaying the final media message (e.g., of the group of media), thecomputer system deletes (e.g., after a predefined duration of time haspassed) the first, second, and third media message (and, optionally thefourth media message) from the computer system (or makes the mediamessage otherwise inaccessible to the user).

Displaying a replay user interface object (e.g., a replay button) toreplay the media message of the group of media message allows the userto replay (and re-listen to, re-watch) the media message without needingto individually select each media message to play back. Providingadditional control options without cluttering the UI with additionaldisplayed controls enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In accordance with some embodiments, the first media message, the secondmedia message, and the third media message (and, optionally, the fourthmedia message) were directed to a target group of devices (e.g., aplurality of devices of a home (e.g., a home automation system)),including the computer system. In accordance with some embodiments, thecomputer system (e.g., 700) displays, via the display generationcomponent, a multicast affordance (e.g., 736) in the media userinterface that, when activated (e.g., 1550 f 2, detecting a tap on),initiates a process for sending an audio multicast message to the targetgroup of devices. In some embodiments, the process for sending the audiomulticast message includes displaying the user interface of FIG. 7C1 andis described with respect to FIG. 7C1. In some embodiments, the one ormore devices are electronic devices that are part of a group ofelectronic device to which the first, second, and third media messagewere directed.

Displaying a multicast affordance to initiate a process for sending anaudio multicast message to one or more devices provides the user withthe ability to respond to the media message that include messageswithout the need to individually select which users/devices will receivethe message, thereby reducing the number of inputs the user needs toprovide to respond. Reducing the number of inputs needed to perform anoperation enhances the operability of the device and makes theuser-device interface more efficient (e.g., by helping the user toprovide proper inputs and reducing user mistakes whenoperating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some embodiments, the process for sending the audio multicast messageincludes recording, at the computer system, an audio message. In someembodiments, the audio message is made available for playback at thecomputer system (e.g., as part of the group of media). In someembodiments, the computer system deletes the first, second, and thirdmedia message (and, optionally the fourth media message) from thecomputer system (or makes the media message otherwise inaccessible tothe user) in conjunction with sending the audio multicast message.

In accordance with some embodiments, while playing a respective mediamessage (e.g., the first media message, the second media message), thecomputer system (e.g., 700) displays, via the display generationcomponent and in the media user interface, a name of a respective user(e.g., 1524 a, 1524 b. 1524 c. 1524 d) from which the respective mediamessage was received. In some embodiments, as the media message that isplaying changes, the name of the respective user is replaced (e.g., theprevious name, the current name, the next name are displayed at the samelocation).

Displaying a name of a respective user from which the currently playingmedia was received provides the user with feedback about the source fromwhich the computer system received the currently playing media message.Providing improved visual feedback to the user enhances the operabilityof the device and makes the user-device interface more efficient (e.g.,by helping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In accordance with some embodiments, a representation of a currentlyplaying media message (e.g., 1524 a in FIG. 15B, 1524 b in FIG. 15C) isdisplayed with a first background color (e.g., green) that is differentfrom the background color of respective displayed representations (e.g.,1524 b and 1524 c in FIG. 15B, 1524 c and 1524 d in FIG. 15C) of mediamessage that are not currently playing.

In accordance with some embodiments, the media message are audiomessages (e.g., multicast audio messages). In some embodiments, theaudio message (e.g., first media message) is an audio message recorded(using a microphone) at an external device and transmitted by theexternal device to a group of devices and/or an audio message that wasdirected to (e.g., transmitted to) a plurality of external devices thatincludes the computer system). In some embodiments, the second mediamessage is a second audio message that was directed (or transmitted) toa second plurality of external devices (e.g., same as the plurality ofexternal devices, different from the plurality of external devices).

Automatically playing back multiple audio messages of a conversation(e.g., in order) enables the computer system to output a conversation(e.g., an audio conversation among multiple participants) withoutrequiring the user to individually select particular audio messages toplay back, thereby reducing the number of required inputs. Reducing thenumber of inputs needed to perform an operation enhances the operabilityof the device and makes the user-device interface more efficient (e.g.,by helping the user to provide proper inputs and reducing user mistakeswhen operating/interacting with the device) which, additionally, reducespower usage and improves battery life of the device by enabling the userto use the device more quickly and efficiently.

In some embodiments, a total count (e.g., “4” in “1 of 4” of 930 in FIG.15B) of media message in the group of media is displayed. In someembodiments, (e.g., along with the total count) a number (e.g., “1” in“1 of 4” of 930 in FIG. 15B, in the ordered set of the group of media)of the currently playing media is displayed (e.g., “3 of 4” as in FIG.15D).

In some embodiments, a media message (e.g., the first media message)corresponds to a multi-user device (e.g., a device that is configuredfor multiple users, a device that is not configured for a particularuser). In some embodiments, a media message (e.g., the second mediamessage) corresponds to a single-user device (e.g., a device that isconfigured for a single user). In some embodiments, a visual indicator(e.g., 1522 a, 1524 a, 1522 c, 1524 c) of a corresponding user of amedia message is displayed concurrent with playing the media messagecorresponding to the user. In some embodiments, a visual indictor (e.g.,1522 b, 1524 b) that indicates a multi-user device is displayed whileplaying the media message corresponding to the multi-user device.

Note that details of the processes described above with respect tomethod 1600 (e.g., FIG. 16) are also applicable in an analogous mannerto the methods described above. For example, method 1600 optionallyincludes one or more of the characteristics of the various methodsdescribed above with reference to methods 800, 1000, and 1400. Foranother example, the audio message of method 800 can be the same as theaudio message sent in method 1600. For another example, the displayednotifications described with respect to method 800 can be the samenotification displayed in method 1600. For another example, voice inputtransmitted to one or more devices with respect to method 1200 can bethe same audio message received and played at the computer system ofmethod 1600. For brevity, these details are not repeated above.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the techniques and their practical applications. Othersskilled in the art are thereby enabled to best utilize the techniquesand various embodiments with various modifications as are suited to theparticular use contemplated.

Although the disclosure and examples have been fully described withreference to the accompanying drawings, it is to be noted that variouschanges and modifications will become apparent to those skilled in theart. Such changes and modifications are to be understood as beingincluded within the scope of the disclosure and examples as defined bythe claims.

As described above, one aspect of the present technology is thegathering and use of data available from various sources to improve thedelivery to users of visual content (e.g., visual indications) or anyother content that provides the user with information concerning whethervoice input will be sent as an audio messages to one or more devices.The present disclosure contemplates that in some instances, thisgathered data may include personal information data that uniquelyidentifies or can be used to contact or locate a specific person.

The present disclosure recognizes that the use of personal informationdata, in the present technology, can be used to the benefit of users.For example, the personal information data can be used to improve thedisplay of an indication so that a respective user can determine whethervoice input will be transmitted as an audio message. Accordingly, use ofsuch personal information data enables users to have calculated controlover sending audio messages. Further, other uses for personalinformation data that benefit the user are also contemplated by thepresent disclosure.

The present disclosure contemplates that the entities responsible forthe collection, analysis, disclosure, transfer, storage, or other use ofsuch personal information data will comply with well-established privacypolicies and/or privacy practices. In particular, such entities shouldimplement and consistently use privacy policies and practices that aregenerally recognized as meeting or exceeding industry or governmentalrequirements for maintaining personal information data private andsecure. Such policies should be easily accessible by users, and shouldbe updated as the collection and/or use of data changes. Personalinformation from users should be collected for legitimate and reasonableuses of the entity and not shared or sold outside of those legitimateuses. Further, such collection/sharing should occur after receiving theinformed consent of the users. Additionally, such entities shouldconsider taking any needed steps for safeguarding and securing access tosuch personal information data and ensuring that others with access tothe personal information data adhere to their privacy policies andprocedures. Further, such entities can subject themselves to evaluationby third parties to certify their adherence to widely accepted privacypolicies and practices. In addition, policies and practices should beadapted for the particular types of personal information data beingcollected and/or accessed and adapted to applicable laws and standards,including jurisdiction-specific considerations. For instance, in the US,collection of or access to certain health data may be governed byfederal and/or state laws, such as the Health Insurance Portability andAccountability Act (HIPAA); whereas health data in other countries maybe subject to other regulations and policies and should be handledaccordingly. Hence different privacy practices should be maintained fordifferent personal data types in each country.

Despite the foregoing, the present disclosure also contemplatesembodiments in which users selectively block the use of, or access to,personal information data. That is, the present disclosure contemplatesthat hardware and/or software elements can be provided to prevent orblock access to such personal information data. For example, in the caseof receiving voice input from a user, the present technology can beconfigured to allow users to select to “opt in” or “opt out” ofparticipation in the collection of personal information data duringregistration for services or anytime thereafter. In another example,users can choose not to provide voice input to be sent as an audiomessage based on the indication that is displayed. In yet anotherexample, users can select to limit the content of the audio message thatcorresponds to the voice input based on the indication that isdisplayed. In addition to providing “opt in” and “opt out” options, thepresent disclosure contemplates providing notifications relating to theaccess or use of personal information. For instance, a user may benotified upon downloading an app that their personal information datawill be accessed and then reminded again just before personalinformation data is accessed by the app.

Moreover, it is the intent of the present disclosure that personalinformation data should be managed and handled in a way to minimizerisks of unintentional or unauthorized access or use. Risk can beminimized by limiting the collection of data and deleting data once itis no longer needed. In addition, and when applicable, including incertain health related applications, data de-identification can be usedto protect a user's privacy. De-identification may be facilitated, whenappropriate, by removing specific identifiers (e.g., date of birth,etc.), controlling the amount or specificity of data stored (e.g.,collecting location data a city level rather than at an address level),controlling how data is stored (e.g., aggregating data across users),and/or other methods.

Therefore, although the present disclosure broadly covers use ofpersonal information data to implement one or more various disclosedembodiments, the present disclosure also contemplates that the variousembodiments can also be implemented without the need for accessing suchpersonal information data. That is, the various embodiments of thepresent technology are not rendered inoperable due to the lack of all ora portion of such personal information data. For example, the visualindication can be displayed to users based on non-personal informationdata or a bare minimum amount of personal information, such as thecontent being requested by the device associated with a user, othernon-personal information available to the audio message transmittingservices, or publicly available information.

What is claimed is:
 1. A computer system, comprising: one or moreprocessors, wherein the computer system is in communication with adisplay generation component and one or more input devices; and memorystoring one or more programs configured to be executed by the one ormore processors, the one or more programs including instructions for:displaying, via the display generation component, a first statusindicator that indicates that a plurality of accessory devices of afirst accessory type is in a first state, wherein the plurality ofaccessory devices of the first accessory type includes: a firstaccessory device that is the first accessory type and is in the firststate; and a second accessory device that is the first accessory typeand is in the first state, wherein the second accessory device isdifferent from the first accessory device; while displaying the firststatus indicator, detecting a gesture; and in response to detecting thegesture: in accordance with a determination that the gesture is a firsttype of gesture and is directed to the first status indicator, causingthe first accessory device and the second accessory device to change toa second state that is different from the first state; and in accordancewith a determination that the gesture is a second type of gesture thatis different from the first type of gesture and is directed to the firststatus indicator: in accordance with a determination that the firststatus indicator corresponds to less than a predetermined number ofaccessory devices of the first accessory type that are in the firststate, displaying a first accessory control user interface objectcorresponding to the first accessory device, wherein selection of thefirst accessory control user interface object causes the first accessorydevice to change to a third state; and in accordance with adetermination that the first status indicator corresponds to not lessthan the predetermined number of accessory devices of the firstaccessory type that are in the first state, displaying a secondaccessory control user interface object corresponding to the firstaccessory device, wherein selection of the second accessory control userinterface object causes the first accessory device to change to a fourthstate that is different from the third state.
 2. The computer system ofclaim 1, wherein: the gesture is detected while concurrently displaying,with the first status indicator, a second status indicator thatindicates that a plurality of accessory devices of a second accessorytype, different from the first accessory type, are in a fifth state, theplurality of accessory devices of the second accessory type including: athird accessory device that is the second accessory type and is in thefifth state; and a fourth accessory device that is the second accessorytype and is in the fifth state; and the one or more programs furtherincludes instructions for: in accordance with a determination that thegesture is the first type of gesture and is directed to the secondstatus indicator, causing the third accessory device and the fourthaccessory device to change to a sixth state that is different from thefifth state; and in accordance with a determination that the gesture isthe second type of gesture and is directed to the second statusindicator, displaying, via the display generation component, a thirdaccessory control user interface object corresponding to the thirdaccessory device, wherein selection of the third accessory control userinterface object causes the third accessory device to change to thesixth state without causing the fourth accessory device to change to thesixth state.
 3. The computer system of claim 1, wherein: in accordancewith a determination that a fifth accessory device is the firstaccessory type and the fifth accessory device is in the first state, theplurality of accessory devices of the first accessory type includes thefifth accessory device; and in accordance with a determination that thefifth accessory device is the first accessory type and the fifthaccessory device is not in the first state, the plurality of accessorydevices of the first accessory type does not include the fifth accessorydevice.
 4. The computer system of claim 1, wherein: the first type ofgesture is a tap gesture; and the second type of gesture is apress-and-hold gesture.
 5. The computer system of claim 1, wherein theone or more programs further include instructions for: in response todetecting the gesture: in accordance with a determination that thegesture is a third type of gesture, displaying a status indicator thatwas not previously displayed.
 6. The computer system of claim 1, whereincausing the first accessory device and the second accessory device tochange to the second state includes causing the first accessory deviceand the second accessory device to perform a type of action that isbased on the first accessory type.
 7. The computer system of claim 1,wherein the one or more programs further include instructions for:before detecting the gesture, displaying a third status indicatoradjacent to the first status indicator and a fourth status indicatoradjacent to the third status indicator, wherein the first statusindicator, the third status indicator, and fourth status indicator aredisplayed in an order based on a respective status indicator categorythat corresponds to each respective status indicator.
 8. The computersystem of claim 1, wherein the one or more programs further includeinstructions for: in accordance with a determination that the gesture isthe first type of gesture and is directed to the first status indicator,ceasing to display the first status indicator after a predeterminedperiod of time.
 9. The computer system of claim 1, wherein: the firststatus indicator is displayed at a first location when the gesture isdetected; and the one or more programs further include instructions for:in accordance with a determination that the gesture is the first type ofgesture and is directed to the first status indicator, ceasing todisplay the first status indicator at the first location and displayinga sixth status indicator at the first location, wherein the sixth statusindicator was adjacent to the first status indicator before the gesturewas detected.
 10. The computer system of claim 1, wherein the one ormore programs further include instructions for: in accordance with thedetermination that the gesture is the first type of gesture and isdirected to the first status indicator, visually updating the firststatus indicator such that the first status indicator is changed fromindicating that the plurality of accessory devices of the firstaccessory type are in the first state to indicate that the plurality ofaccessory devices of the first accessory type has changed state.
 11. Thecomputer system of claim 1, wherein: the first status indicator isdisplayed on a first user interface; and the one or more programsfurther include instructions for: receiving a request to display asecond user interface that is different from the first user interface;and in response to receiving the request to display the second userinterface: displaying the second user interface that includes a statusindicator that indicates that one or more accessory devices of the firstaccessory type are in the first state, wherein: in accordance with adetermination that the second user interface corresponds to a firstgroup of accessory devices and the first accessory device corresponds tothe first group of accessory devices, the one or more accessory devicesof the first accessory type include the first accessory device; and inaccordance with a determination that the second user interfacecorresponds to the first group of accessory devices and the firstaccessory device does not correspond to the first group of accessorydevices, the one or more accessory devices of the first accessory typedoes not include the first accessory device.
 12. The computer system ofclaim 1, wherein the one or more programs further include instructionsfor: in accordance with a determination that the gesture is the secondtype of gesture and is directed to the first status indicator,concurrently, via the display generation component, displaying: a fourthaccessory control user interface object corresponding to the firstaccessory device, wherein selection of the fourth accessory control userinterface object causes the first accessory device to change to thesecond state without causing the second accessory device to change tothe second state; and a fifth accessory control user interface objectcorresponding to the second accessory device, wherein selection of thefifth accessory control user interface object causes the secondaccessory device to change to the second state without causing the firstaccessory device to change to the second state.
 13. A non-transitorycomputer-readable storage medium storing one or more programs configuredto be executed by one or more processors of a computer system, whereinthe computer system is in communication with a display generationcomponent and one or more input devices, the one or more programsincluding instructions for: displaying, via the display generationcomponent, a first status indicator that indicates that a plurality ofaccessory devices of a first accessory type is in a first state, whereinthe plurality of accessory devices of the first accessory type includes:a first accessory device that is the first accessory type and is in thefirst state; and a second accessory device that is the first accessorytype and is in the first state, wherein the second accessory device isdifferent from the first accessory device; while displaying the firststatus indicator, detecting a gesture; and in response to detecting thegesture: in accordance with a determination that the gesture is a firsttype of gesture and is directed to the first status indicator, causingthe first accessory device and the second accessory device to change toa second state that is different from the first state; and in accordancewith a determination that the gesture is a second type of gesture thatis different from the first type of gesture and is directed to the firststatus indicator: in accordance with a determination that the firststatus indicator corresponds to less than a predetermined number ofaccessory devices of the first accessory type that are in the firststate, displaying a first accessory control user interface objectcorresponding to the first accessory device, wherein selection of thefirst accessory control user interface object causes the first accessorydevice to change to a third state; and in accordance with adetermination that the first status indicator corresponds to not lessthan the predetermined number of accessory devices of the firstaccessory type that are in the first state, displaying a secondaccessory control user interface object corresponding to the firstaccessory device, wherein selection of the second accessory control userinterface object causes the first accessory device to change to a fourthstate that is different from the third state.
 14. The non-transitorycomputer-readable storage medium of claim 13, wherein: the gesture isdetected while concurrently displaying, with the first status indicator,a second status indicator that indicates that a plurality of accessorydevices of a second accessory type, different from the first accessorytype, are in a fifth state, the plurality of accessory devices of thesecond accessory type including: a third accessory device that is thesecond accessory type and is in the fifth state; and a fourth accessorydevice that is the second accessory type and is in the fifth state; andthe one or more programs further include instructions for: in accordancewith a determination that the gesture is the first type of gesture andis directed to the second status indicator, causing the third accessorydevice and the fourth accessory device to change to a sixth state thatis different from the fifth state; and in accordance with adetermination that the gesture is the second type of gesture and isdirected to the second status indicator, displaying, via the displaygeneration component, a third accessory control user interface objectcorresponding to the third accessory device, wherein selection of thethird accessory control user interface object causes the third accessorydevice to change to the sixth state without causing the fourth accessorydevice to change to the sixth state.
 15. The non-transitorycomputer-readable storage medium of claim 13, wherein: in accordancewith a determination that a fifth accessory device is the firstaccessory type and the fifth accessory device is in the first state, theplurality of accessory devices of the first accessory type includes thefifth accessory device; and in accordance with a determination that thefifth accessory device is the first accessory type and the fifthaccessory device is not in the first state, the plurality of accessorydevices of the first accessory type does not include the fifth accessorydevice.
 16. The non-transitory computer-readable storage medium of claim13, wherein: the first type of gesture is a tap gesture; and the secondtype of gesture is a press-and-hold gesture.
 17. The non-transitorycomputer-readable storage medium of claim 13, wherein the one or moreprograms further include instructions for: in response to detecting thegesture: in accordance with a determination that the gesture is a thirdtype of gesture, displaying a status indicator that was not previouslydisplayed.
 18. The non-transitory computer-readable storage medium ofclaim 13, wherein causing the first accessory device and the secondaccessory device to change to the second state includes causing thefirst accessory device and the second accessory device to perform a typeof action that is based on the first accessory type.
 19. Thenon-transitory computer-readable storage medium of claim 13, wherein theone or more programs further include instructions for: before detectingthe gesture, displaying a third status indicator adjacent to the firststatus indicator and a fourth status indicator adjacent to the thirdstatus indicator, wherein the first status indicator, the third statusindicator, and fourth status indicator are displayed in an order basedon a respective status indicator category that corresponds to eachrespective status indicator.
 20. The non-transitory computer-readablestorage medium of claim 13, wherein the one or more programs furtherinclude instructions for: in accordance with a determination that thegesture is the first type of gesture and is directed to the first statusindicator, ceasing to display the first status indicator after apredetermined period of time.
 21. The non-transitory computer-readablestorage medium of claim 13, wherein: the first status indicator isdisplayed at a first location when the gesture is detected; and the oneor more programs further include instructions for: in accordance with adetermination that the gesture is the first type of gesture and isdirected to the first status indicator, ceasing to display the firststatus indicator at the first location and displaying a sixth statusindicator at the first location, wherein the sixth status indicator wasadjacent to the first status indicator before the gesture was detected.22. The non-transitory computer-readable storage medium of claim 13,wherein the one or more programs further include instructions for: inaccordance with the determination that the gesture is the first type ofgesture and is directed to the first status indicator, visually updatingthe first status indicator such that the first status indicator ischanged from indicating that the plurality of accessory devices of thefirst accessory type are in the first state to indicate that theplurality of accessory devices of the first accessory type has changedstate.
 23. The non-transitory computer-readable storage medium of claim13, wherein: the first status indicator is displayed on a first userinterface; and the one or more programs further include instructionsfor: receiving a request to display a second user interface that isdifferent from the first user interface; and in response to receivingthe request to display the second user interface: displaying the seconduser interface that includes a status indicator that indicates that oneor more accessory devices of the first accessory type are in the firststate, wherein: in accordance with a determination that the second userinterface corresponds to a first group of accessory devices and thefirst accessory device corresponds to the first group of accessorydevices, the one or more accessory devices of the first accessory typeinclude the first accessory device; and in accordance with adetermination that the second user interface corresponds to the firstgroup of accessory devices and the first accessory device does notcorrespond to the first group of accessory devices, the one or moreaccessory devices of the first accessory type does not include the firstaccessory device.
 24. The non-transitory computer-readable medium ofclaim 13, wherein the one or more programs further include instructionsfor: in accordance with a determination that the gesture is the secondtype of gesture and is directed to the first status indicator,concurrently, via the display generation component, displaying: a fourthaccessory control user interface object corresponding to the firstaccessory device, wherein selection of the fourth accessory control userinterface object causes the first accessory device to change to thesecond state without causing the second accessory device to change tothe second state; and a fifth accessory control user interface objectcorresponding to the second accessory device, wherein selection of thefifth accessory control user interface object causes the secondaccessory device to change to the second state without causing the firstaccessory device to change to the second state.
 25. A method,comprising: at a computer system, wherein the computer system is incommunication with a display generation component and one or more inputsdevices: displaying, via the display generation component, a firststatus indicator that indicates that a plurality of accessory devices ofa first accessory type is in a first state, wherein the plurality ofaccessory devices of the first accessory type includes: a firstaccessory device that is the first accessory type and is in the firststate; and a second accessory device that is the first accessory typeand is in the first state, wherein the second accessory device isdifferent from the first accessory device; while displaying the firststatus indicator, detecting a gesture; and in response to detecting thegesture: in accordance with a determination that the gesture is a firsttype of gesture and is directed to the first status indicator, causingthe first accessory device and the second accessory device to change toa second state that is different from the first state; and in accordancewith a determination that the gesture is a second type of gesture thatis different from the first type of gesture and is directed to the firststatus indicator: in accordance with a determination that the firststatus indicator corresponds to less than a predetermined number ofaccessory devices of the first accessory type that are in the firststate, displaying a first accessory control user interface objectcorresponding to the first accessory device, wherein selection of thefirst accessory control user interface object causes the first accessorydevice to change to a third state; and in accordance with adetermination that the first status indicator corresponds to not lessthan the predetermined number of accessory devices of the firstaccessory type that are in the first state, displaying a secondaccessory control user interface object corresponding to the firstaccessory device, wherein selection of the second accessory control userinterface object causes the first accessory device to change to a fourthstate that is different from the third state.
 26. The method of claim25, wherein: the gesture is detected while concurrently displaying, withthe first status indicator, a second status indicator that indicatesthat a plurality of accessory devices of a second accessory type,different from the first accessory type, are in a fifth state, theplurality of accessory devices of the second accessory type including: athird accessory device that is the second accessory type and is in thefifth state; and a fourth accessory device that is the second accessorytype and is in the fifth state; and the method further comprises: inaccordance with a determination that the gesture is the first type ofgesture and is directed to the second status indicator, causing thethird accessory device and the fourth accessory device to change to asixth state that is different from the fifth state; and in accordancewith a determination that the gesture is the second type of gesture andis directed to the second status indicator, displaying, via the displaygeneration component, a third accessory control user interface objectcorresponding to the third accessory device, wherein selection of thethird accessory control user interface object causes the third accessorydevice to change to the sixth state without causing the fourth accessorydevice to change to the sixth state.
 27. The method of claim 25,wherein: in accordance with a determination that a fifth accessorydevice is the first accessory type and the fifth accessory device is inthe first state, the plurality of accessory devices of the firstaccessory type includes the fifth accessory device; and in accordancewith a determination that the fifth accessory device is the firstaccessory type and the fifth accessory device is not in the first state,the plurality of accessory devices of the first accessory type does notinclude the fifth accessory device.
 28. The method of claim 25, wherein:the first type of gesture is a tap gesture; and the second type ofgesture is a press-and-hold gesture.
 29. The method of claim 25, furthercomprising: in response to detecting the gesture: in accordance with adetermination that the gesture is a third type of gesture, displaying astatus indicator that was not previously displayed.
 30. The method ofclaim 25, wherein causing the first accessory device and the secondaccessory device to change to the second state includes causing thefirst accessory device and the second accessory device to perform a typeof action that is based on the first accessory type.
 31. The method ofclaim 25, further comprising: before detecting the gesture, displaying athird status indicator adjacent to the first status indicator and afourth status indicator adjacent to the third status indicator, whereinthe first status indicator, the third status indicator, and fourthstatus indicator are displayed in an order based on a respective statusindicator category that corresponds to each respective status indicator.32. The method of claim 25, further comprising: in accordance with adetermination that the gesture is the first type of gesture and isdirected to the first status indicator, ceasing to display the firststatus indicator after a predetermined period of time.
 33. The method ofclaim 25, wherein: the first status indicator is displayed at a firstlocation when the gesture is detected; and the method further comprises:in accordance with a determination that the gesture is the first type ofgesture and is directed to the first status indicator, ceasing todisplay the first status indicator at the first location and displayinga sixth status indicator at the first location, wherein the sixth statusindicator was adjacent to the first status indicator before the gesturewas detected.
 34. The method of claim 25, further comprising: inaccordance with the determination that the gesture is the first type ofgesture and is directed to the first status indicator, visually updatingthe first status indicator such that the first status indicator ischanged from indicating that the plurality of accessory devices of thefirst accessory type are in the first state to indicate that theplurality of accessory devices of the first accessory type has changedstate.
 35. The method of claim 25, wherein: the first status indicatoris displayed on a first user interface; and the method furthercomprises: receiving a request to display a second user interface thatis different from the first user interface; and in response to receivingthe request to display the second user interface: displaying the seconduser interface that includes a status indicator that indicates that oneor more accessory devices of the first accessory type are in the firststate, wherein: in accordance with a determination that the second userinterface corresponds to a first group of accessory devices and thefirst accessory device corresponds to the first group of accessorydevices, the one or more accessory devices of the first accessory typeinclude the first accessory device; and in accordance with adetermination that the second user interface corresponds to the firstgroup of accessory devices and the first accessory device does notcorrespond to the first group of accessory devices, the one or moreaccessory devices of the first accessory type does not include the firstaccessory device.
 36. The method of claim 25, further comprising: inaccordance with a determination that the gesture is the second type ofgesture and is directed to the first status indicator, concurrently, viathe display generation component, displaying: a fourth accessory controluser interface object corresponding to the first accessory device,wherein selection of the fourth accessory control user interface objectcauses the first accessory device to change to the second state withoutcausing the second accessory device to change to the second state; and afifth accessory control user interface object corresponding to thesecond accessory device, wherein selection of the fifth accessorycontrol user interface object causes the second accessory device tochange to the second state without causing the first accessory device tochange to the second state.